System for accurate object detection with multiple sensors

ABSTRACT

A system and method for providing safety for a motorized mobile system (MMS) are optimized for accurate object detection, the system comprising at least two sensors that are operably configured to generate one or more of a range measurement and a bearing measurement to an object, wherein the range and bearing measurements have associated uncertainties.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/880,699, entitled System and Methods for Sensor Integrationin Support of Situational Awareness for a Motorized Mobile System, filedJan. 26, 2018, which claims priority to U.S. Provisional Patent App. No.62/543,896, entitled Systems and Methods for Motorized Mobile Systems,filed Aug. 10, 2017, and U.S. Provisional Patent App. No. 62/612,617,entitled Systems and Methods for Enhanced Autonomous Operations of aMotorized Mobile System, filed Dec. 31, 2017, which are incorporatedherein by reference in their entirety. The present application isrelated to U.S. patent application Ser. No. 16/434,000, entitled Systemand Methods for Sensor Integration in Support of Situational Awarenessfor a Motorized Mobile System, filed on the same date as the presentapplication, which is incorporated herein by reference in its entirety.The present application also is related to U.S. patent application Ser.No. 15/880,663, entitled Secure Systems Architecture for IntegratedMotorized Mobile Systems, filed Jan. 26, 2018, U.S. patent applicationSer. No. 15/880,686, entitled Federated Sensor Array for Use with aMotorized Mobile System and Method of Use, filed Jan. 26, 2018, all ofwhich are incorporated herein by reference in their entirety.

COPYRIGHT NOTICE

Contained herein is material that is subject to copyright protection.The copyright owner has no objection to the facsimile reproduction byanyone of the patent document or the patent disclosure, as it appears inthe United States Patent and Trademark Office patent file or records,but otherwise reserves all rights to the copyright whatsoever. Thefollowing notice applies to the software, screenshots and data asdescribed below and in the drawings hereto and All Rights Reserved.

FIELD

This disclosure relates generally to control systems and sensor systemsfor motorized mobile systems.

BACKGROUND

Drive-by-wire (DbW), steer-by-wire, or x-by-wire technology is the useof electrical or electro-mechanical systems for performing vehiclefunctions traditionally achieved by mechanical linkages. This technologyreplaces the traditional mechanical control systems with electroniccontrol systems using electromechanical actuators and human-machineinterfaces. The technology is similar to the fly-by-wire systems used inthe aviation industry. Use of these “by-wire” systems began with mannedaircraft, migrated to drones, as well as marine and rail operations, andare now being used in autonomous or self-driving vehicle applications.These once expensive technologies are emerging in the market ascommodity products, including products with sensors, processors,integrated mobile devices, and various communication mediums, includingbandwidth increases for soon to be 5^(th) generation (5G) wirelessdevices on 5G networks.

This application and co-pending applications will create and achievesafe, secure independence and a richer experience for all motorizedmobile system (MMS) users. As an example of the need for improved MMSs,consider that today, with the advances in robotics and systems ofsystems integration, as well as medical advances that allow deviceintegration with the human nervous system, there is a widening splitbetween MMS users with varying physiological functionality. Some mobilechair users may have significant remaining upper body mobility andcognitive function. An example of this would be a person who does nothave the use of their legs and who uses a manual mobile chair formobility, but is otherwise able to navigate day-to-day life with minimalto no assistance. Such an individual may be able to adapt to anartificial limb, such as a leg, or an exoskeleton and reasonably be ableto go about their day to day life with few restrictions. However,another example would be a user with certain health issues that greatlyimpacts the user's mobility and/or cognition. It is unlikely that theseusers will benefit from the same artificial leg or exoskeletontechnologies due to their physiological condition. These users may use amotorized mobile system, such as a mobile chair.

Many mobile chair users report they are frequently frustrated by thegeneral public's poor understanding of their abilities and needs. Ingeneral, the mobile chair is an extension of a user's body. People whouse them have different disabilities and varying abilities. Some can usetheir arms and hands, while others can get out of their mobile chairsand walk for short distances. “Disability” is a general, medical termused for a functional limitation that interferes with a person's abilityto walk, hear, learn, or utilize other physiological and/or cognitivefunctions of the body.

Conditions like cerebral palsy can be a sub-set of either physiologicalor cognitive disabilities since there are a number of sub-typesclassified based on specific ailments they present, resulting in varyingdegrees of ability. For example, those with stiff muscles have what ismedically defined as spastic cerebral palsy, those with poorcoordination have ataxic cerebral palsy, and those with writhingmovements have athetoid cerebral palsy, each type requiring individualmobility plans.

Following are a few definitions used in this disclosure.

People with disabilities: This term represents a universe of potentialconditions, including physical, cognitive, and/or sensory conditions.

Mobility disability: This term represents a condition for a person whouses a mobile chair or other MMS to assist in mobility.

User: This term refers to an individual who uses an MMS. A “user” of amobile chair is referred to herein as a “mobile chair user”.

Operator: This term refers to an individual who operates an MMS,including manual, local, and remote operation.

Caregiver: This term represents any individual that assists an MMS user.Family, friends, aides, and nurses may all be included in this category.The term “Attendant” is used synonymously with the term caregiver.

Technician: This term includes one or more of those individuals whosetup, service, modify, or otherwise work technically on an MMS. Theseindividuals may be formally licensed or may include operators andcaregivers who are comfortable working with the system.

A mobile chair is essentially a chair with wheels used when walking isdifficult or impossible due to illness, injury, or disability. Mobilechairs come in a wide variety to meet the specific needs of their users,including:

-   -   Manual self-propelled mobile chairs.    -   Manual attendant-propelled mobile chairs.    -   Powered mobile chairs (power-chairs).    -   Mobility scooters.    -   Single-arm drive mobile chairs.    -   Reclining mobile chairs.    -   Standing mobile chairs.    -   Combinations of the above.

Mobile Chairs include specialized seating adaptions and/orindividualized controls and may be specific to particular activities.The most widely recognized distinction in mobile chairs is powered andunpowered. Unpowered mobile chairs are propelled manually by the user orattendant while powered mobile chairs are propelled using electricmotors.

Motorized mobile chairs are useful for those unable to propel a manualmobile chair or who may need to use a mobile chair for distances or overterrain which would be fatiguing or impossible in a manual mobile chair.They may also be used not just by people with ‘traditional’ mobilityimpairments, but also by people with cardiovascular and fatigue-basedconditions. A Motorized Mobile System (MMS) is a non-automobilemotorized device which provides powered mobility to one or more users,including such systems as powered mobile chairs, mobility scooters,electronic conveyance vehicles, riding lawn mowers, grocery carts,all-terrain vehicles (ATVs), golf carts, and other recreational and/ormedical mobility systems, but excludes automobiles (passenger cars,trucks, passenger buses, and other passenger or property transportingmotorized vehicles intended for licensed operation on state and nationalhighways). For the sake of clarity, a mobile chair MMS is describedherein as an exemplary embodiment; however, it should be clear that thesame or similar systems and methods may be applied to other MMSembodiments.

A mobile chair MMS is generally four-wheeled or six-wheeled andnon-folding. Four general styles of mobile chair MMS drive systemsexist: front, center, rear, and all-wheel drive. Powered wheels aretypically somewhat larger than the trailing/castering wheels, whilecastering wheels on a motorized chair are typically larger than thecasters on a manual chair. Center wheel drive mobile chair MMSs havecasters at both front and rear for a six-wheel layout and are oftenfavored for their tight turning radii. Front wheel drive mobile chairMMSs are often used because of their superior curb-climbingcapabilities. Power-chair chassis may also mount a specificcurb-climber, a powered device to lift the front wheels over a curb of10 cm or less.

Mobile chair MMSs are most commonly controlled by arm-rest mountedjoysticks which may have additional controls to allow the user to tailorsensitivity or access multiple control modes, including modes for theseating system. For users who are unable to use a hand controller,various alternatives are available, such as sip-and-puff controllers,worked by blowing into a sensor. In some cases, a controller may bemounted for use by an aide walking behind the chair rather than by theuser. Capabilities include turning one drive-wheel forward while theother goes backward, thus turning the mobile chair within its ownlength.

The seating system on a mobile chair MMS can vary in design, including abasic sling seat and backrest, optional padding, comfortable cushions,backrest options, and headrests. Many companies produce aftermarketseat, back, leg, and head rest options which can be fitted onto mobilechair MMSs. Some seat, back, leg, and head rests are produced to aidwith increased need for stability in the trunk or for those at increasedrisk of pressure sores from sitting. Leg rests may be integrated intothe seating design and may include manual and/or powered adjustment forthose users who want or need to vary their leg position. Mobile chairMMSs may also have a tilt-in-space, or reclining facility, which isparticularly useful for users who are unable to maintain an uprightseating position indefinitely. This function can also help with comfortby shifting pressure to different areas over time, or with positioningin a mobile chair when a user needs to get out of the chair or behoisted.

Most mobile chairs are crash tested to ISO standards 7176 and 10542.These standards mean that a mobile chair can be used facing forward in avehicle if the vehicle has been fitted with an approved tie down ordocking system for securing the mobile chair and a method of securingthe occupant to the mobile chair.

Rehabilitation engineering is the systematic application of engineeringsciences to design, develop, adapt, test, evaluate, apply, anddistribute technological solutions to problems confronted by individualswith disabilities. Current practitioners of rehabilitation engineeringare often forced to work with limited information and make long termdecisions about the technologies to be used by an individual on thebasis of a single evaluation; a snapshot in time. Under currentbest-case conditions, rehabilitation engineering practitioners workclosely in a long-term relationship with their clients to follow-up andreadjust assistive technology systems on a regular basis. However, evenin these situations, they are often working with limited information andonly at periodic intervals.

What is needed is an evolution of existing motorized mobile systems(MMSs) to consider the users' abilities, needs, and health, with thegoal of a safe, secure, and social independence. To accomplish this,systems and methods are disclosed herein comprising: integrated softwareand hardware systems, sensors for situational awareness, sensors foruser monitoring, communications between users and caregivers, users andother users, and users and the “cloud”, and human machine interfaces(HMIs) designed for users with a variety of physiological and cognitiveconditions. The systems and methods disclosed herein are based on newunderlying technologies, architectures, and network topologies thatsupport the evolution of the MMS.

SUMMARY

Four co-pending-applications disclose various aspects of improved MMSs.All four are disclosed as related above and each incorporates byreference herein in the entirety of the other applications in full.

The application entitled “Secure Systems Architecture for MotorizedMobile Systems,” relates to systems and methods for implementing acontrol system onboard an MMS capable of securely communicating with andutilizing external systems. This may include integrating externaldevices and user health monitoring sensors with an off the shelf (OTS)or custom MMS. Integration of a smart device, such as a smart phone ortablet, with an OTS or custom MMS is another example. Today, most smartdevices contain a host of applications and sensors, including one ormore of image capturing devices, rate and acceleration sensors,gyroscopes, global positioning system (GPS) receivers, biometricsensors, iris scanners, fingerprint scanners, and facial recognitionsoftware. Other sensors are possible. A secure architecture for an MMScontroller is disclosed in support of device integration and datasecurity with a focus on extensibility.

The application entitled “Federated Sensor Array for Use with aMotorized Mobile System and Method of Use” discloses the integration ofnon-contact sensors and control logic into an MMS controller. Thefederated sensors have overlapping sensing fields, generally operateindependently, and report certain data relevant to navigation andstability which is then used by the MMS controller. Motor, seat, andauxiliary controllers may be hosted in the MMS controller along with thefederated sensor logic. The integration of these systems andapplications into an MMS lays the foundation for situational awareness(SA).

Situational awareness is the ability to be cognizant of oneself in agiven space. It is an organized knowledge of objects and statekinematics in relation to oneself in a given space or scenario.Situational awareness also involves understanding the relationship ofthese objects when there is a change of position or kinematic state. Thegoal is to integrate this data into the MMS and use it to support aricher, safer, and more independent experience for the user.

The application entitled “System and Methods for Sensor Integration inSupport of Situational Awareness for a Motorized Mobile System” furtherdiscloses the integration of new sensor technologies in support of adeeper and richer situational awareness for the user. These new systemsuse the data generated about the user, the environment, targets in theenvironment, and the user's relationship to them. This information maybe generated from one or more sources and include data from non-contactsensors, like radar, optical, laser, and ultrasonic sensors. Thesenon-contact sensors can generate data about the environment, includingrange measurements, bearing measurements, target classification, andtarget kinematics. The new sensors provide a much richer set of dataabout the environment.

The federated system uses a single type of sensor that generates asingle report (i.e. a communication with or identifying data sensed bythe sensor) with what is called a single mode variance, where eachsensor has distinct capabilities and one or more fixed errors inherentto the sensor. Ultra-sonic sensors have better range determination thancross range position determination, for instance. In an example, usingdata from a different type of sensor, a good cross range report can begenerated, but with poor down range determination. In this evolvingsystem, the best of two (or more) separate reports may be combined. Thisis referred to as a dual mode variance.

The application entitled “System and Methods for Enhanced AutonomousOperations of a Motorized Mobile System” discloses the implementation ofadvanced filtering techniques and sensor fusion in support ofsituational awareness and autonomy. Adding more sensors to a federationof sensors increases expense, weight, and power consumption. Integrationand use of sensor fusion (e.g. using different types of sensors in onesystem) and advanced filtering techniques improves the information theMMS controller uses to track the user and environment, while reducingcomplexity and cost when compared to a federated approach. Decisionlogic consisting of data association techniques, track and targetmanagement, handling out of sequence measurements, and sensor framemanagement are all building blocks for this leap in system capability.

In this enhanced system, raw data is received and “filtered”, or as isknown in the art fused, with other data related to the MMS user andtheir activities while navigating in the environment. The other data mayinclude certain biometric data, user inputs, and user activities.Filtering and state estimation are some of the most pervasive tools ofengineering. In some embodiments, a model may be used to form aprediction of a state into the future, followed by an observation of thestate or actual measurement of the expectation. A comparison of thepredicted state and the measured state is then made. If the observationsmade are within the predicted measurements, the model may be adjusted byreducing the covariance of the next measurement, thereby increasingsystem confidence. If the observations are outside of the predictedmeasurements, the model may be adjusted to increase the covariance ofthe next measurement, thereby decreasing system confidence.

In this enhanced system, the MMS is fully aware of its environment andcan travel safely wherever the user wishes to go, within reason.Moreover, the MMS may learn to anticipate the needs of the user. Theresult is a user experience that is safe, secure, and independent, basedon the user's base abilities and current condition.

Other systems may be integrated to improve user experience. As anon-limiting example, augmented reality (AR) may be included. Augmentedreality is a live direct or indirect view of a physical, real-worldenvironment where the elements are augmented (or supplemented) bycomputer-generated sensory input. The input can be sound, smell, orgraphics. It is related to a more general concept calledcomputer-mediated reality, in which a view of reality is modified,possibly even diminished rather than augmented, by a computer. As aresult, the technology functions by enhancing one's current perceptionof reality. Virtual Reality (VR) is another technology that may beintegrated to improve user experience. By contrast, VR replaces the realworld with a simulated one. Augmentation is conventionally in real timeand in semantic context with environmental elements, such as sportsscores on TV during a match. However, VR refers to computer technologiesthat use VR headsets, sometimes in combination with physical spaces ormulti-projected environments, to generate realistic images, sounds, andother sensations that simulate a user's physical presence in a virtualor imaginary environment.

In one aspect, a motorized mobile system (MMS) comprises a first sensor,a second sensor, and a processor. The first sensor generates firstsensor data about an object, the first sensor data about the objectcomprising a first range measurement to the object and a first bearingmeasurement to the object, the first range measurement having anassociated first uncertainty, and the first bearing measurement havingan associated second uncertainty. The second sensor generates secondsensor data about the object, the second sensor data about the objectcomprising a second range measurement to the object and a second bearingmeasurement to the object, the second range measurement having anassociated third uncertainty, and the second bearing measurement havingan associated fourth uncertainty. The processor receives the firstsensor data about the object, receives the second sensor data about theobject, selects a lower range uncertainty between the first uncertaintyand the third uncertainty, selects a lower bearing uncertainty betweenthe second uncertainty and the fourth uncertainty, and combines thebearing measurement associated with the selected lower bearinguncertainty and the range measurement associated with the selected lowerrange uncertainty as a location of the object within a reduced area ofuncertainty.

In another aspect, a motorized mobile system (MMS) comprises a firstsensor, a second sensor, and a processor. The first sensor generatesfirst sensor data about an object, the object located proximate to themotorized mobile system, wherein the first sensor data about the objectcomprises a first range measurement to the object and a first bearingmeasurement to the object, the first range measurement has an associatedfirst uncertainty, and the first bearing measurement has an associatedsecond uncertainty. The second sensor generates second sensor data aboutthe object, the object located proximate to the motorized mobile system,wherein the second sensor data about the object comprises a second rangemeasurement to the object and a second bearing measurement to theobject, the second range measurement has an associated thirduncertainty, and the second bearing measurement has an associated fourthuncertainty. The processor receives the first sensor data about theobject, receives the second sensor data about the object, selects alower range uncertainty between the first uncertainty and the thirduncertainty, selects a lower bearing uncertainty between the seconduncertainty and the fourth uncertainty, and combines the bearingmeasurement associated with the selected lower bearing uncertainty andthe range measurement associated with the selected lower rangeuncertainty as a location of the object within a reduced area ofuncertainty.

Applicant(s) herein expressly incorporate(s) by reference all of thefollowing materials identified in each paragraph below. The incorporatedmaterials are not necessarily “prior art”.

ISO/IEC 15408-1:2009, 3rd Edition: “Information technology—Securitytechniques—Evaluation criteria for IT security—Part 1: Introduction andgeneral model”.

ISO/IEC 15408-2:2008, 3rd Edition: “Information technology—Securitytechniques—Evaluation criteria for IT security—Part 2: Securityfunctional components”.

ISO/IEC 15408-3:2008, 3rd Edition: “Information technology—Securitytechniques—Evaluation criteria for IT security—Part 3: Securityassurance components”.

802.11-2016: “IEEE Standard for Informationtechnology—Telecommunications and information exchange between systemsLocal and metropolitan area networks—Specific requirements—Part 11:Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY)Specifications”.

1609.0-2013: “IEEE Guide for Wireless Access in Vehicular Environments(WAVE)—Architecture”.

1609.2-2016: “IEEE Standard for Wireless Access in VehicularEnvironments—Security Services for Applications and ManagementMessages”.

1609.4-2016: “IEEE Standard for Wireless Access in VehicularEnvironments (WAVE)—Multi-Channel Operation”.

1609.11-2010: “IEEE Standard for Wireless Access in VehicularEnvironments (WAVE)—Over-the-Air Electronic Payment Data ExchangeProtocol for Intelligent Transportation Systems (ITS)”.

1609.12-2016: “IEEE Standard for Wireless Access in VehicularEnvironments (WAVE)—Identifier Allocations”.

ETSI EN 302 663 (V1.2.1): “Intelligent Transport Systems (ITS); Accesslayer specification for Intelligent Transport Systems operating in the 5GHz frequency band.”

ETSI EN 302 571 (V1.2.1): “Intelligent Transport Systems (ITS);Radiocommunications equipment operating in the 5 855 MHz to 5 925 MHzfrequency band; Harmonized EN covering the essential requirements ofarticle 3.2 of the R&TTE Directive”.

ETSI TS 102 792 (V1.2.1): “Intelligent Transport Systems (ITS);Mitigation techniques to avoid interference between European CENDedicated Short Range Communication (CEN DSRC) equipment and IntelligentTransport Systems (ITS) operating in the 5 GHz frequency range”.

IEEE 802-2014: “IEEE Standard for Local and Metropolitan Area Networks:Overview and Architecture”.

ANSI/IEEE Std 802.2 (1998): “IEEE Standard for Informationtechnology—Telecommunications and information exchange betweensystems—Local and metropolitan area networks—Specific requirements—Part2: Logical Link Control”.

ISO/IEC 7498-1:1994: “Information technology—Open SystemsInterconnection—Basic Reference Model: The Basic Model”.

ITU-T Recommendation X.691 (2015): “Information technology—ASN.1encoding rules: Specification of Packed Encoding Rules (PER)”.

ETSI TS 102 687 (V1.1.1): “Intelligent Transport Systems (ITS);Decentralized Congestion Control Mechanisms for Intelligent TransportSystems operating in the 5 GHz range; Access layer part”.

IEEE 1003.1-2008: “IEEE Standard for Information Technology—PortableOperating System Interface (POSIX®)”.

IEEE 802.15.1-2005: “Wireless medium access control (MAC) and physicallayer (PHY) specifications for wireless personal area networks (WPANs)”.

IEEE 802.15.4-2015: “IEEE Standard for Low-Rate Wireless Networks”.

ISO/IEC 18092:2013: “Information technology—Telecommunications andinformation exchange between systems—Near Field Communication—Interfaceand Protocol (NFCIP-1)”.

IEEE 802.16-2012: “IEEE Standard for Air Interface for BroadbandWireless Access Systems”.

ISO/IEEE 11073-20601-2014: “IEEE Health informatics—Personal healthdevice communication—Part 20601: Application profile—Optimized ExchangeProtocol”.

Bluetooth SIG: “Bluetooth Core Specification”, v5.0.

If it is believed that any of the above-incorporated materialconstitutes “essential material” within the meaning of 37 CFR1.57(d)(1)-(3), applicant(s) reserve the right to amend thespecification to expressly recite the essential material that isincorporated by reference as allowed by the applicable rules.

Aspects and applications presented here are described below in thedrawings and detailed description. Unless specifically noted, it isintended that the words and phrases in the specification and the claimsbe given their plain and ordinary meaning to those of ordinary skill inthe applicable arts. The inventors are aware that they can be their ownlexicographers if desired. The inventors expressly elect, as their ownlexicographers, to use only the plain and ordinary meaning of terms inthe specification and claims unless they clearly state otherwise andexpressly set forth the “special” definition of that term. Absent suchclear statements of intent to apply a “special” definition, it is theinventors' intent and desire that the plain and ordinary meaning to theterms be applied to the interpretation of the specification and claims.

Further, the inventors are informed of the standards and application ofthe special provisions of 35 U.S.C. § 112(f). Thus, the use of the words“function,” “means” or “step” in the Detailed Description or Descriptionof the Drawings or claims is not intended to somehow indicate a desireto invoke the special provisions of 35 U.S.C. § 112(f) to define thesystems, methods, processes, and/or apparatuses disclosed herein. To thecontrary, if the provisions of 35 U.S.C. § 112(f) are sought to beinvoked to define the embodiments, the claims will specifically andexpressly state the exact phrases “means for” or “step for” and willalso recite the word “function” (i.e., will state “means for performingthe function of . . . ”), without also reciting in such phrases anystructure, material, or act in support of the function. Thus, even whenthe claims recite a “means for performing the function of . . . ” or“step for performing the function of . . . ”, if the claims also reciteany structure, material, or acts in support of that means or step thenit is the clear intention of the inventors not to invoke the provisionsof 35 U.S.C. § 112(f). Moreover, even if the provisions of 35 U.S.C. §112(f) are invoked to define the claimed embodiments, it is intendedthat the embodiments not be limited only to the specific structures,materials, or acts that are described in the preferred embodiments, butin addition, include any and all structures, materials, or acts thatperform the claimed function as described in alternative embodiments orforms, or that are well known present or later-developed equivalentstructures, materials, or acts for performing the claimed function.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the systems, methods, processes, and/orapparatuses disclosed herein may be derived by referring to the detaileddescription when considered in connection with the followingillustrative figures. In the figures, like-reference numbers refer tolike-elements or acts throughout the figures.

FIG. 1 depicts a control system architecture with hardware and softwarecomponents for an S-MMS.

FIG. 2 depicts an embodiment of an S-MMS hardware architecture.

FIG. 3 depicts an embodiment of a control system architecture for anS-MMS hosting an integrated situational awareness controller.

FIG. 4 depicts an embodiment of a control system architecture for anS-MMS with a situational awareness controller.

FIG. 5 depicts an embodiment of a situational awareness controller.

FIG. 6 depicts an embodiment of a sensor system.

FIG. 7 depicts an example embodiment of 360-degree situationalawareness.

FIG. 8 depicts reference frames of typical sensors.

FIG. 9 depicts an S-MMS frame of reference.

FIG. 10A depicts a navigation frame of reference.

FIG. 10B depicts another view of the navigation frame of reference ofFIG. 10A.

FIG. 10C depicts an actual navigation frame of reference and a perceivednavigation frame of reference.

FIG. 11 depicts an embodiment of an actual navigation frame of referencethat deviates from a perceived navigation frame of reference.

FIG. 12 depicts two sensors operating with different fields of view.

FIG. 13 depicts a standard conversion of error (variance).

FIG. 14 depicts two S-MMS users approaching a blind intersection of twopathways.

FIG. 15 depicts an embodiment of an S-MMS system securely connected to aremote server.

FIG. 16A depicts an embodiment of an S-MMS communicating with GPSsatellites.

FIG. 16B depicts an embodiment of an S-MMS navigating an indoorenvironment based on one or more sensors without GPS information.

FIG. 17 depicts one or more S-MMSs connected to a remote server.

FIG. 18 depicts multiple S-MMS users combining map data.

FIG. 19 depicts multiple users uploading accessibility information to ashared map repository.

FIG. 20 depicts an embodiment wherein an S-MMS uses onboard sensors tosense and model an object.

FIG. 21 depicts an S-MMS paired to an augmented reality (AR) device viaa wireless link.

FIG. 22 depicts a typical path and motions required to successfully parka mobile chair MMS inside an accessible van.

FIG. 23 depicts use of targets to assist in S-MMS positioning.

FIG. 24 depicts a common S-MMS transfer situation.

FIG. 25 depicts an embodiment of approach mode on an S-MMS.

FIG. 26A depicts an exemplary placement of sensors to measure forces onan arm rest.

FIG. 26B depicts an exemplary placement of sensors to measure forces ona foot rest.

FIG. 27A depicts an embodiment of a self-retracting control unit whenextended.

FIG. 27B depicts an embodiment of a self-retracting control unit whenretracted.

FIG. 28A depicts an S-MMS traveling to its charging station.

FIG. 28B depicts an S-MMS returning to the user.

FIG. 29 depicts an embodiment in which a wearable device with anembedded transceiver communicates with an S-MMS.

FIG. 30 depicts an example of a service animal “nudging” an S-MMS andredirecting it.

FIG. 31A depicts an S-MMS orienting a user to face individuals with whomthe user is engaged.

FIG. 31B depicts an S-MMS orienting a user to face individuals with whomthe user is engaged.

FIG. 32A depicts an example of crowd interaction with an S-MMS sensorsystem.

FIG. 32B depicts an example of crowd interaction with an S-MMS sensorsystem.

FIG. 33 depicts crowd navigation enhanced by integration of a navigationsystem.

FIG. 34 depicts an embodiment of auto-queuing functionality.

Elements and acts in the figures are illustrated for simplicity and havenot necessarily been rendered according to any particular sequence orembodiment.

DETAILED DESCRIPTION

In the following description, and for the purposes of explanation,numerous specific details, process durations, and/or specific formulavalues are set forth in order to provide a thorough understanding of thevarious aspects of exemplary embodiments. However, it will be understoodby those skilled in the relevant arts that the apparatus, systems, andmethods herein may be practiced without all of these specific details,process durations, and/or specific formula values. Other embodiments maybe utilized and structural and functional changes may be made withoutdeparting from the scope of the apparatus, systems, and methods herein.It should be noted that there are different and alternativeconfigurations, devices, and technologies to which the disclosedembodiments may be applied. The full scope of the embodiments is notlimited to the examples that are described below.

In the following examples of the illustrated embodiments, references aremade to the accompanying drawings which form a part hereof, and in whichis shown by way of illustration various embodiments in which thesystems, methods, processes, and/or apparatuses disclosed herein may bepracticed. It is to be understood that other embodiments may be utilizedand structural and functional changes may be made without departing fromthe scope.

Systems and methods are disclosed for use of multiple sensors and theirassociated data on a smart motorized mobile system (S-MMS). Referringgenerally to FIGS. 1-34, systems, methods, and apparatuses for providingsafety and independence for S-MMS users are illustrated. The systems andmethods disclosed support non-automobile motorized mobile systems, suchas powered mobile chairs, mobility scooters, electronic conveyancevehicles, riding lawn mowers, grocery carts, ATVs, golf carts, off-roadvehicles, and other recreational and/or medical mobility systems, butexcludes automobiles. For the purposes of this disclosure, automobilesare defined as passenger cars, trucks, passenger buses, and otherpassenger or property transporting motorized vehicles intended forlicensed operation on state and national highways. A non-limiting,illustrative example of a motorized mobile chair is used throughout thedisclosure. In various embodiments, a placement or location of at leastone sensor may be determined based at least in part upon unique S-MMSdynamic characteristics, user seating position, wheel or track locationassociated with the S-MMS, or other characteristics relevant to thedisclosed systems and methods.

Some embodiments of the systems disclosed may be referred to as separategenerations based on level of capability. While these generations arediscussed as separate embodiments, it should be clear that any one ormore aspects of any one or more generations may be combined to formother systems not explicitly disclosed herein (or in the relatedco-pending applications). The generations are as follows: Generation 0(Gen 0), Generation I (Gen I), Generation II (Gen II), and GenerationIII (Gen III).

The motorized mobile systems in existence today may be referred toherein as Generation 0. Generation 0 is an MMS with a user interface anda control system. Generation 0 is hosted in a controller with a HumanMachine Interface (HMI) typically consisting of a joystick, tactilesurface array, sip and puff type array, or similar interface. Thejoystick receives input indicating “move forward”, the command isgenerated, and control instructions are sent to a motor controller,which responds with a preconfigured response. The control instructionsmay include a change in state or an adjustment of an operatingparameter. The state of the art for a Generation 0 system is to provideextremely simple control instructions and open loop limits on the MMS.Open loop systems lack the ability for self-correcting actions. Anexample of an open loop limit currently in use on MMSs is to cut themaximum MMS speed to a predetermined set point if the user raises theseat position above a certain threshold. The motor controller respondsdirectly to the user input regardless of the environment proximate tothe MMS. A new user may have a learning curve to master before they canconfidently maneuver close to people, objects, or in confinedenvironments.

A Smart Motorized Mobile System (S-MMS) of the present disclosure is anevolution of MMS technology, and includes embodiments of a newcontroller and control architecture, some of which include securemethods for collecting and transmitting data across one or morenetworks.

The present application and one or more related applications discloseimproved generations of S-MMS architectures, including Generations I-IIIarchitectures. Generation I is an embodiment for a group of sensorsreporting to a controller for the S-MMS. Generation II embodimentsfurther include consideration for scanning and/or image sensorsoperating in overlapping regions. Using a sensor with good down rangeerror, and a second sensor with good cross range error, a Generation IIsystem embodiment can coordinate reports in real-time, associate them,and take the best measurements in an ability to make the situationalawareness picture more accurate. This use of more than one sensor istypically referred to as dual mode variance.

Generation II S-MMSs may take advantage of their ability to exchangedata with other like equipped systems about their environment. Inaddition to other like equipped systems, the S-MMS may be configured toreceive data from traffic through Dedicated Short-Range Communications(DSRC) across an IEEE 802.11P link. This data may help the user tobetter navigate next to a road way, or along paths that are shared byautomobiles and other MMSs. For an S-MMS user, the ability to announceone's presence and the ability to control traffic could be lifesaving.

Generation II systems are based on historical data or on observations ofstate at a finite time, in this case time T₁. In one example, an eventis measured at time T₁, the event is processed at time T₂, data for theprocessed event is transmitted at time T₃, the data for the processedevent is related to other data at time T₄, and any other actions thatneed to be carried out are done at time T₅. This can be done veryquickly, e.g. from tenths of a second to even seconds. Regardless ofdelay, it is all historic.

Generation III is an embodiment for a multi-generation controllerarchitecture and logic. In some embodiments, a Generation III system mayhost one or more of the previous generations or combinations thereof.Generation III systems go beyond dual mode variance to true sensorfusion.

The S-MMS controller may be one or more processors (hardware),application-specific integrated circuits, or field-programmable gatearrays that host the disclosed architecture. Control signals may be viawired or wireless communications, and comprised of digital and/or analogsignals.

Control System Embodiment

FIG. 1 depicts an embodiment of an S-MMS 18 control system composed ofhardware and software components. An S-MMS controller 110 lies betweentwo secure abstraction layers 135 and 145. Abstraction layers are usedas a way of hiding the implementation details of a particular set offunctionalities, allowing the separation of concerns to facilitateinteroperability and platform independence. The upper abstraction layer135 abstracts through an Application Programmers Interface (API) to ahosted application space 125 and a complex application space 130. TheAPI is a set of subroutine definitions, protocols, and tools forbuilding applications. An API allows for communication between thevarious components. The complex application space 130 may include hostedcontrol logic for an S-MMS 18. Below the S-MMS controller 110 and itssecure abstraction 145 is a breakout of the operating system 150, one ormore processors 160 (which are hardware), and a communications layer170, which may include a hardware communications interface. Memory 120,which is hardware, cross cuts all of the layers in the depictedembodiment and may include volatile and non-volatile non-transitorycomputer storage media for storing information. The S-MMS controller 110is software that executes on one or more processors 160 on the S-MMS 18and is stored in memory 120.

With a focus now on the one or more hardware processors that the S-MMScontroller 110 is executed on and interacts with, FIG. 2 depicts ahardware embodiment of an S-MMS 18A architecture. The depictedelectrical architecture comprises an S-MMS processor 202 between twosecurity processors 204 and 212, each of which is hardware. The S-MMSprocessor 202 may be paired with a lock-step processor (not depicted)for critical life, health, and safety applications, in some embodiments.The security processors 204 and 212 may host (i.e. execute) modules andother software, such as the previously disclosed secure abstraction APIs135 and 145. Additionally or alternatively, the security processors mayhost services, such as watch-dog and data source authenticationservices. The S-MMS controller 110A is hosted on an S-MMS processor 202.The processors may comprise one or more of a processor, multipleprocessors, an application-specific integrated circuit, or afield-programmable gate array.

The S-MMS controller 110A utilizes computer readable storage media 220,which includes the memory 120, for data storage and retrieval duringoperation. Executable program instructions for the S-MMS controller 210Dalso may be stored in the memory 120. The memory 120 is one or more of avolatile and non-volatile non-transitory computer storage medium forstoring information and may be located onboard the S-MMS 18A, may beremote storage available on a smart device or server, or somecombination of the foregoing. One or more secure, encrypted memorypartitions are used to store electronic protected health information(ePHI) and other secure health data. The data stored on the securememory is only made available to one or more pre-authorized systems,wherein the pre-authorized system comprises a device or serviceassociated with an individual user. This may include a mobile motorizedsystem, a smart device, a computer, a data terminal, or a device orservice associated with an approved third party.

The S-MMS 18A hardware system may comprise multiple additionalprocessors beyond the core S-MMS processor 202. In the case of a powerwheelchair S-MMS, these additional hardware processors may include oneor more caregiver processors 206, one or more HMI processors 208, one ormore application processors 210, one or more sensor processors 214, oneor more communication processors 216, and one or more drive processors218, each of which is hardware. Each processor executes software and mayproduce control signals wherein the control signal is a wired orwireless signal, and wherein the control signal comprises one or more ofa digital or an analog signal, and generally comprises or indicatesdata, instructions, and/or a state. A brief description of each of theadditional processors for the depicted embodiment is provided below.

A caregiver processor 206 may be physically attached to the S-MMS or maybe part of a remote device. In one embodiment, a caregiver processor 206is a duplicate HMI and associated processor for the S-MMS that allows acaregiver to physically drive or otherwise maneuver or control the S-MMSor its components.

An HMI processor 208 may accept one or more user inputs from one or moreHMI devices, such as a joystick or touch screen, and convert them intoone or more control signals with data and/or instructions which aretransmitted in response to the one or more user inputs at the HMI.Control instructions may comprise one or more of a calculation, alogical comparison, a state, a change in state, an instruction, arequest, data, a sensor reading or record, an adjustment of an operatingparameter, a limitation of a feature or capability, or an enablement ofa feature or capability.

An application processor 210 may include one or more processors embeddedin ancillary products, such as a seat controller, lighting controller,or 3^(rd) party device. Typically, these processors receive one or morecontrol signals that causes them to respond with a preconfiguredresponse, wherein the preconfigured response may include moving,measuring, changing a state, transmitting data, or taking operationalcontrol of the associated hardware (e.g. raising, lowering, or angling aseat or increasing or decreasing a light brightness or turning a lighton or off). An application processor 210 may additionally oralternatively supply data about the S-MMS or use data generated from oneor more sensors.

A sensor processor 214 receives data generated from one or more sensorsused by the S-MMS or otherwise associated with one or morecharacteristics of the mobile system or a user of the mobile system. Thereceived data may be stored in a memory and/or transmitted. Multiplesensors may use a single sensor processor 214 or multiple processors.Additionally or alternatively, individual sensors may have their own(e.g. dedicated) processors.

A communication processor 216 is used to establish one or moreconnections with one or more devices and transmits communications to,and receives communications from, one or more devices through associateddevices of the S-MMS (e.g. one or more transceivers). Devices maycommunicate with the processor via wired or wireless means. Thesedevices may be located on the S-MMS 18A or may be remote to the S-MMS18A. A communication processor 216 may be part of a communication systemfor a mobile system for secure transmission and/or secure reception ofdata. In some embodiments, the S-MMS processor 202 may have anintegrated communication processor or the S-MMS processor performs thefunctions of the communication processor

In an exemplary embodiment, a communication processor 216 on the S-MMS18A is configured to establish secure connections between the S-MMS 18Aand one or more other wireless devices over which data is transmittedand received by the communication processor and the one or more wirelessdevices. Responsive to a secure connection being established by thecommunication processor 216 with a wireless device, the communicationprocessor retrieves from a secure memory 220 one or more of stored firstdata or stored second data; wherein first data is data generated fromone or more sensors associated with one or more characteristics of themobile system (e.g. sensors on or used by the S-MMS 18A for measurementof distances, angles, or planes at which the S-MMS is operating, drivespeed or direction, angular momentum, or other operationalcharacteristics of the S-MMS itself) and second data is data generatedfrom one or more sensors associated with a user of the mobile system(e.g. user presence in the seat, heart rate, seat moisture, or othercharacteristics of the user of the S-MMS). One or more of the first dataand second data is then communicated to the wireless device via thesecure connection for storage in a secure second memory of the wirelessdevice. The wireless device associated and the communication processor216 may communicate using one or more of cellular, 802.11, Wi-Fi,802.15, Bluetooth, Bluetooth Low Energy, 802.20, and WiMAX.

A drive processor 218 receives one or more control signals, for examplefrom the S-MMS controller 110A, that cause the drive processor torespond with a preconfigured response to the steering system and/ordrive motor(s) of the S-MMS, wherein the preconfigured response includesone or more of taking operational control of the steering system ordrive motor(s), steering the S-MMS, or starting and/or stopping one ormore drive motors to move the S-MMS in one or more directions. A driveprocessor 218 may additionally or alternatively supply data generatedfrom one or more sensors associated with one or more characteristics ofthe mobile system to the S-MMS controller 110A.

In some embodiments, one or more sensors may be mounted to differentphysical locations on an S-MMS 18A. In some embodiments, the sensingarea/view/field of one or more sensors may overlap the sensingarea/view/field of one or more other sensors or a contiguous sensingfield may exist between sensors to obtain a complete 360-degree sensingarea view around the S-MMS 18A, which is referred to herein as afederation of sensors. In some embodiments, the one or more sensors arenon-cooperating independent sensors that generate a detection responseto objects with some confidence (e.g. generate a control signal thatindicates one or more objects were detected and a distance to the one ormore objects or other measurement data relative to the one or moreobjects). In such an embodiment, the kinematic states of detection thatcan be determined include position and time of detection. In someembodiments, control logic may be deployed in an S-MMS controller 110Ato create an integrated system of systems within the S-MMS 18A.

Situational Awareness System

Situational awareness (SA) is the perception of environmental elements,objects, conditions, and events with respect to the observer, in termsof time and space. Situational awareness involves being aware of what ishappening in the vicinity of the user to understand how information,events, and one's own actions will impact objectives, both immediatelyand in the near future. More importantly, situational awareness involvesthe comprehension of information's meaning, and the projection of anobject's status after some variable has changed or occurred, such astime, or some other variable has changed or occurred, such as apredetermined event. Situational awareness is also the field of studyconcerned with understanding the environment critical to adecision-making process in a complex dynamic system. Dynamic situationsmay include ordinary, but nevertheless complex, tasks such asmaneuvering an S-MMS in an environment safely.

A federation of sensors may be oriented proximate to an S-MMS 18 in sucha way to ensure 360-degree sensor coverage. These sensors may report toan S-MMS controller 110 that is tasked with receiving information fromone or more sensors, interpreting information from one or more sensors,and taking action based on information provided by the one or moresensors. The S-MMS controller 110 may attempt to assign meaning to theinformation and create a projection of one or more statuses or states ofthe S-MMS 18, including after some variable has changed or occurred.When one or more sensors report data identifying an object, the S-MMScontroller 110 may respond based on those reports, either automaticallyor with user input.

FIG. 3 depicts an embodiment in which the S-MMS controller 110B isdeployed on an S-MMS. The depicted embodiment comprises a situationalawareness controller 302 within the S-MMS controller 110B. The S-MMSsituational awareness controller 302 communicates with a motorcontroller 351 and a Human Machine Interface (HMI) 352. In someembodiments, one or more of the motor controller 351 and HMI 352 may beintegrated into the S-MMS controller 110B. The depicted S-MMS controller110B further comprises real-time operating system (RTOS) services 362and navigation 363.

An arbitration Information Assurity Manager (IAM) 370 manages sensorreports from one or more sensors on or used by the S-MMS 18 and mayinclude communication, navigation, and identification (CNI) 371processing capabilities. Communications received from a sensor aretermed sensor reports. In some embodiments, the arbitration IAM 370resides on a security or arbitration processor 212 (FIG. 2).Additionally or alternatively, functions of the CNI 371 may be performedby a dedicated communication processor 216 (FIG. 2). Sensor reportsreceived and managed by the arbitration IAM 370 may include non-contactsensor reports 372, search and track sensor reports 373, image sensorreports 374, and user sensor reports 375. Sensor reports (372-375) maybe composed of data stored in one or more of long-term or short-termsystem memory 120 or read from an input port on an S-MMS 18A processor(e.g. processors 212, 216 or 214). A report (371-376) may includeadditional data beyond a simple measurement, including sensor status,confidence levels, or other information.

Non-contact sensors are devices used to take a measurement, often adistance, without coming in contact with the detected object. There aremany types of non-contact sensors, including optical (e.g. LIDAR),acoustic (e.g. RADAR or ultrasonic), and magnetic (e.g. hall effectsensor). Microphones may additionally be included as a non-contactsensor. Search and track sensors may include image and non-contactsensor types, but are sensors that often have larger fields of view andmay scan within these fields of view. Image sensors detect and conveyinformation that constitutes an image or series of images/video, whereinthe image(s)/video may contain light or electromagnetic radiationinformation on an area. These sensor reports interface to the specificsensor types in the system to identify measurements, detections, number,efficiency, health, degraded performance, states, statuses, and/or otherdata of each sensor in the sensing system.

The depicted arbitration IAM 370 further comprises a global positioningsystem (GPS) and inertial manager 376. In the depicted embodiment, thesituational awareness controller 302 communicates with the CNI 371,sensor reports 372, 373, 374, and 375 and navigation 363. Navigation 363communicates with the GPS and inertial manager 376 in the arbitrationIAM 370. The depicted embodiment of the situational awareness controller302 includes logic to manage the sensors, including one or more of onand off, sweep rate, sensor volume, regional interrogation, and/or otheroperations.

The CNI 371 manages communications through system links and off boardlinks to enable vehicle to device, intra-vehicle, and inter-vehiclecommunication and coordination, including cooperative navigation amongvehicles and using other devices and identification of devices andvehicles. In some embodiments, the CNI 371 identifies other data sourcesand retrieves data from other data sources, including for threatsdetected and kinematic states of sensors, vehicles, and devices. The CNI371 is also responsible for GPS corrected system-wide time and processortime sync across the system in conjunction with the operating system.For example, the CNI 371 receives an accurate time via the GPS andinertial manager 376 and transmits that accurate time to all hardwareprocessors along with an instruction to sync their internal clocks tothat accurate time. This time coordination function is important in someembodiments since errors in time coordination can introduce as mucherror in system performance as a bad sensor reading in thoseembodiments. Propagating a sensor measurement to the wrong point in timecan induce significant confusion to filters, such as a Kalman filter,especially if time goes backward due to sync errors.

The CNI 371, in some embodiments, may be configured to receive data fromone or more different sources, including other like-equipped S-MMSs,vehicles and traffic devices, among others, through a DedicatedShort-Range Transceiver (DSRC), for instance, across an IEEE 802.11Plink, which may be formatted in an IEEE 1609 format. This data may helpthe user to better navigate next to a roadway or along paths that areshared by automobiles. Some embodiments may allow for traffic lights,speed signs, and traffic routing to be dynamically altered. For an S-MMSuser, the ability to announce one's presence and thereby enable atraffic control device to effect a change in traffic, such as bychanging a stop light to red, could be lifesaving.

A search and track function 373 may be used to maintain sensor health,detect sensor failures, monitor sensor zones of coverage (sensor zones),and notify the situational awareness controller 302 or other componentof the S-MMS controller 110B of sensor system degradation or otherstates. The search and track function 373 may also manage the transitionof sensors from online and off-line states (including plug and playfuture options).

The user sensor reports 375, in some embodiments, may be configured toreceive data from one or more sensors used to monitor user condition,user position, and/or user status. This data may allow for assistivebehaviors to be triggered and/or tuned by the situational awarenesscontroller 302 to the human in the loop of the S-MMS 18.

The GPS and inertial manager 376 includes one or more inertialmeasurement unit (IMU) data services that receives one or more reportsfrom an attitude and heading reference system (AHRS) that includes anIMU. An IMU of an AHRS consists of one or more sensors on three axesthat provide attitude information of the S-MMS 18 to the GPS andinertial manager 376, including yaw, pitch, and roll of the S-MMS anddeviations to each. As an example, an x-axis may typically be lateralacross the S-MMS 18 coaxial with the axles of the front wheels of theS-MMS, extending 90-degrees left and 90-degrees right, a y-axis mayextend forward and rearward of the S-MMS, and a z-axis may extendvertically through the S-MMS, 90-degrees to the x and y axes. An IMUtypically comprises acceleration and rate determining sensors on eachaxis. In the case of x, y, and z measurements, the IMU is referred to asa 6-Degree of Freedom (DOF) sensor. Some IMUs also have a small halldevice on each axis to measure the magnetic line of flux of the earth'smagnetic poles, similar to a compass, that allows for the calculation oftrue, earth referenced orientation. These IMU embodiments are referredto as a 9-DOF sensor and are more accurate than a 6-DOF sensor. However,some systems may interpolate the z-axis by detecting gravity on eitherthe x or y axes, which may be less accurate. The IMU is fixed to theS-MMS 18 in some embodiments and provides reports to the GPS andinertial manager 376.

The GPS and inertial manager 376 also receives GPS signals from a GPSreceiver. The GPS receiver may be mounted to the S-MMS 18, be part of asmart device paired or otherwise linked to the S-MMS, or be anotherreceiver that transmits signals to the S-MMS or a smart device linked tothe S-MMS.

Navigation 363, in the depicted embodiment, is an inertial referencesystem, including an inertial navigation system (INS), for navigatingusing dead reckoning (DR). Dead reckoning is the process of calculatingthe S-MMS 18 current position by using a previously determined position,or fix, and advancing that position based upon known or estimated speedsand steering over elapsed time and heading. In one embodiment, DR usesGPS location data (e.g. via GPS and inertial manager 376) to update theINS DR fix. Speed, heading, and elapsed time data are then provided bythe INS function of navigation 363 to the situational awarenesscontroller 302. S-MMS speed (e.g. velocity and/or acceleration) may bereceived directly from one or more motor controllers 351. Additionally,speed and heading may be received or calculated from one or more GPS andinertial manager 376 reports. Elapsed time is provided by RTOS services362. The navigation 363 allows the S-MMS 18 to navigate inside abuilding without GPS or otherwise (e.g. outside of GPS coverage) to asimilar degree of accuracy as navigating outside with continuous GPSdata or otherwise. Speed, heading, and elapsed time for navigation 363,in some other embodiments, is calculated onboard the processors ofinternal and/or external sensors, including one or more GPS receiversand one or more solid state inertial measurement units (IMUs). In someembodiments, the S-MMS processor 202 calculates speed, heading, andelapsed time and generates steering and drive signals, includingoptionally based on one or more non-contact sensor reports and/or GPSand inertial manager reports.

FIG. 4 illustrates an embodiment of a system in which logic 402 for oneor more architectures, including one or more generations referencedabove, is deployed on the control system architecture depicted in FIG.3. The logic 402 may take full advantage of the entire suite of sensorsavailable in the depicted control system embodiment and/or other sensorsnot depicted. The logic 402 processes reports and other data from smartsensors, including smart non-contact sensors 372, search and tracksensors 373, and image sensors 374. These sensors often have largerfields of view and may scan within these fields of view. These sensorsmay be able to characterize reports, i.e. generate both quantitative andqualitative attributes to sensor reports and coordinate sensor reportsin time. These capabilities may be used to support data associationtechniques that can eventually be used in predictive systems.

The complexity and capability of the situational awareness controller302 may dictate the types of applications it will support. In oneexample, the logic 402 supports simple applications, like tip detection,drop off detection, and/or function override for safety. In anotherexample, the logic 402 supports user assistance systems that will resultin a workload reduction for both the user and/or caregiver. In anotherexample, the logic 402 supports increased user independence due toincreased confidence in system actions.

FIG. 5 depicts an embodiment of a situational awareness controller 302Bthat may be deployed in the architecture described by FIG. 4. Theembodiment comprises a sensor track fusion 500, a user health manager510, a sensor tasker 520, a stability manager 525, a collision manager526, a tactical manager 527, a threat assessor 528, a drive path manager529, a feature extraction 530, and an alert manager 540. These processesinteract with the CNI 371, non-contact sensor reports 372, S&T sensorreports 373, image sensor reports 374, user sensor reports 375, andnavigation 363.

Sensor track fusion (STF) 500 is responsible for processing, filtering,and reporting detections of one or more areas around an S-MMS 18 (e.g.between 0 and 360 degrees from a point or location on the S-MMS 18, 360degrees around an S-MMS 18, and/or one or more overlapping areas from oraround the S-MMS 18). This is accomplished by processing reports fromthe available sensors 371-375 and/or feature extraction 530. Next,recursive filtering (e.g. in one embodiment a Kalman filter) is used toestimate the state of each track based on the processed reports (e.g.371-375, 530) in combination with one or more kinematic models (e.g.algorithms or mathematic equations) of possible track motion, forexample, to determine if the track is in motion or stationary relativeto the S-MMS. The estimate is compared to the state of each previouslyidentified track maintained by the tactical manager 527 of the SAC 302B.If the STF 500 determines there is a high-quality match, the state ofthat track is updated in the tactical manager 527. If not, a new trackis generated and communicated to the tactical manager 527. Upon trackgeneration, a priority may be established based on time to impact by thethreat assessor 528, and additional sensor data may be requested bytransmitting a tasking request to sensor tasking 520.

Sensor fusion is the combining data from two or more sensors for thepurpose of improving data quality and/or performance. In the embodimentdepicted in FIG. 5, the boxes around the sensor track fusion 500 anduser health manager 510 are dashed to indicate that all processes arehistorically based. These processes are focused on the creation oftracks and associating current sensor readings in ways that increase theconfidence in the parameters (e.g. such as range and bearing) of thosetracks.

The systems herein are predictive systems, wherein the sensor trackfusion 500 and user health manager 510 processes may be enhanced basedon one or more of:

-   -   a prior knowledge of state,    -   a prediction step based on a physical model,    -   where measurements are made and compared to the measurements,        and,    -   an update step is performed that is either:        -   recursively used as a state transition model becoming the            next prior knowledge of state, or        -   an output estimate of state for the next step.

This approach allows the S-MMS 18 to predict ahead of measurements anamount of time such that when an output estimate of state is generated,the system acts at T₀ or T_(k) from T_(k−1). Essentially, the estimateof state at time step k before the k^(th) measurement has been takeninto account along with the difference between the state transition stepand the predicted step in uncertainty. Uncertainty, for the purpose ofthis disclosure, may be a parameter, associated with the result of ameasurement, that characterizes the dispersion of the values that couldreasonably be attributed to the measurement. In lay terms, theprediction of state is at some probability of accuracy surrounded bysome uncertainty. At some point, the predictions are within theestimates and actions may be taken, essentially at time T₀.

The sensors used with the disclosed control system may operate withlarge fields of view (FOV). These FOVs may typically be in the 30 to120-degree range, in some embodiments. Larger FOVs reduce the number ofsensors needed to achieve situational awareness of the S-MMS 18. In someembodiments, two or more sensors operate in overlapping zones ofcoverage. This overlap allows sensor reports for a single track,condition, or object to be generated by more than one sensor and/orsensors of different types.

A user health manager (UHM) 510 is responsible for processing,filtering, and reporting changes in user behavior and/or health that maybe relevant to the situational awareness controller 302B of the S-MMS.Data from user sensor reports 375, CNI 371, and/or HMI inputs 352 (FIG.4) are used by the UHM 510 to monitor individual and combined healthmetrics. The UHM 510 then determines the appropriate timeline for anyS-MMS controller 110B intervention or adjustments to SAC 302B behavior.The UHM 510 may further access a user profile and log user data tosecure memory linked to the user profile.

A sensor tasker (ST) 520 responds to sensor information requests fromsensor track fusion 500 and sensor information prioritizations from thethreat assessor 528. These priorities are used by the sensor tasker 520to change sensor update rates for possible unsafe conditions (e.g. suchas collisions or tipping danger) and to manage sensor front-endresources to maintain the highest quality tracks on the closestdetections. A track is a threat to the S-MMS 18 that has been identifiedby the SAC 302B and given a unique identifier for use in futurecalculations and processes. The sensor tasker 520 may act as a filterand utilizes the S-MMS system kinematic and identity information fordecision making based on predefined decision boundaries to control theamount of sensor data provided to the SAC 302B at a given instant.

A stability manager 525 determines the stability of the S-MMS 18 andwhether the upcoming ground is likely to cause instability (e.g.tipping) based on one or more inputs from navigation 363, sensor trackfusion 500, and/or threat assessor 528. The stability of the S-MMS 18 iscalculated based on orientation readings (e.g. pitch and/or roll)received from navigation 363 (e.g. from one or more inertial measurementunits fixed to the S-MMS) in combination with a mathematical tippingmodel for the S-MMS 18 stored in memory and executed by the stabilitymanager 525. The stability manager 525 also determines the suitabilityof upcoming terrain based on factors, such as topographic profile and/orsurface composition as measured by one or more sensors 371-374 inrelation to current S-MMS 18 operation characteristics, such asorientation and S-MMS kinematics (e.g. rate of travel, direction oftravel, and S-MMS configuration settings). In an example, an S-MMS 18may have known limits for pitch and roll where tilting beyond the knownlimits causes the S-MMS to tip. Based on the current orientation (e.g.pitch and/or roll) of the S-MMS, the stability manager 525 may usereadings of the ground slope around the S-MMS to estimate the futurepitch/roll of the S-MMS if it travels further forward. If the pitch/rollis below a threshold, the action is allowable. If not, then the actionis not allowable. A list of stability conditions and their locationrelative to the S-MMS 18 are provided to the tactical manager 527 forintegration into the master threat assessment map. The appropriatetimeline for any necessary steering actions and any future coupledsteering maneuvers to minimize instability accidents may be computed bythe stability manager 525. System cutoffs, emergency braking, and/ormotor controller disengagement functions may be performed by thisstability manager 525. The stability manager 525 may further includecrash event data recording and logging for use in S-MMS systemdiagnostics, in some embodiments.

A collision manager 526 computes the Time to Impact (TTI) function basedon one or more inputs from threat assessor 528 and sensor fusion 500.The TTI function uses received sensor data associated with one or moreobjects located around the S-MMS 18 in combination with the currentstate of the S-MMS 18 (e.g. heading, velocity, and/or acceleration) toestimate the time until a collision with those objects is anticipated.The collision manager 526 then determines the appropriate timeline forany necessary steering actions and any future coupled steering maneuversto minimize collision accidents. Any system cutoffs, emergency braking,and/or motor controller disengagement functions may be performed by thecollision manager 526. The collision manager 526 may further includecrash event data recording and logging for use in S-MMS systemdiagnostics, in some embodiments.

A tactical manager 527 uses inputs from one or more of a stabilitymanager 525, collision manager 526, drive path manager 529, navigation363, threat assessor 528, and/or sensor track fusion 500 to maintain amaster, 360-degree map of known objects and conditions in relation tothe S-MMS 18. The tactical manager 527 combines the outputs of thestability manager 525 (e.g. a map of the ground surrounding the S-MMS)and collision manager 526 into a single, integrated map of known tracks.Each track may be assigned a threat level by the threat assessor 528.The current direction and speed of travel (e.g. from navigation 363)and/or desired future direction and speed of travel (e.g. drive pathmanager 529) can then be overlaid on the threat assessment mapmaintained by the tactical manager 527.

A threat assessor 528 function evaluates the multiple tracks and/orobjects detected by the SAC 302B processes, including the stabilitymanager 525, collision manager 526, tactical manager 527, and featureextraction 530, and prioritizes them. Prioritization may be based on arules engine. The rules engine executes one or more predefined rules aspart of threat assessor 528. In one example, the rules engine uses astatistical assessment of the threat posed by each track and/or objectbased on the estimated time to impact to each track provided by thecollision manager 526 in combination with the safe speed determined inthe direction of travel provided by the stability manager 525 and/ordrive path manager 529. If an identified track presents a statisticalrisk above a predefined threshold, the threat assessor 528 willprioritize that track above other tracks with lower calculated risknumbers. The prioritized list of tracks from the threat assessor 528 issent to the sensor tasker 520 which may take additional readings orotherwise focus sensor resources on the highest threat tracks.Additionally, the prioritized list of tracks from the threat assessor528 may be sent to the tactical manager 527 so that areas with a highconcentration of high threat tracks may be flagged as keep-out zones.

A drive path manager (DM) 529 is responsible for route determination,interpretation of external map data (e.g. received from external sourcessuch as a remote server or smart device via CNI 371) for future advancedrouting functions, and generation of steering actions for autonomous bywire speed sensitive steering support. External map data, when received,must be oriented to match the S-MMS 18 reference frame by the DM 529 sothat it can be used with the threat assessment map maintained by thetactical manager 527. The DM 529 combines one or more inputs from thetactical manager 527, sensor track fusion 500, threat assessor 528,and/or navigation 363 in order to determine where the S-MMS 18 shoulddrive. The DM 529 contains a complex 6-DOF S-MMS 18 model that may beused in predictive applications to support stop and go functions, insome embodiments.

Feature extraction 530 performs angle resolution, object detection, edgedetection, and bore sight correction for the S-MMS 18. Angle resolutionis the process of determining the location and error of the orientationof a feature (e.g. object, edge, surface, or plane) around the S-MMS 18.Bore sight correction is the process of correcting downrangemeasurements for sensor misalignment. The sensor tasker 520 receives rawdata of one or more image sensors via the image sensor reports 374, usespixel masks from the sensor tasker 520, produces a detection report byapplying one or more pixel masks to one or more image sensor reports,and assigns a unique identifier (identity) and kinematic data to eachtrack identified in a detection report for sensor track fusion 500consumption. Pixel masks are used to filter out or obscure areas of animage that are not of current concern to the SAC 302B process in orderto decrease compute time and resources required for the process. In someembodiments, feature extraction 530 may be used in a similar way by theuser health manager 510 to measure user health metrics.

In some embodiments, an alert manager 540 may be hosted on (e.g.executed by) the SAC 302B, one of the SAC processors (FIG. 2), and/orhosted in one or more of the application spaces 125 and/or 130 (FIG. 1).

In an embodiment, the alert manager 540 responds to inputs fromstability manager 525, collision manager 526, drive path manager 529,and/or STF 500 to alert the user to possible dangers, enabling awarning, caution, or advisory to actually come from the direction of theproblem. For example, the alert manager 540 of the SAC 302B generatesone or more signals to one or more speakers on or around the S-MMS 18causing the one or more speakers to produce one or more sounds that areeither generic or particular to the particular speakers (e.g. one soundfor a right lateral speaker to notify the user of an alert on the rightlateral side of the S-MMS, another sound for a left lateral speaker tonotify the user of an alert on the left lateral side of the S-MMS 18,another sound for a forward speaker to notify the user of an alert infront of the S-MMS, and another sound for a rear speaker to notify theuser of an alert behind the S-MMS). This alert manager 540 may receiveand process data from the collision manager 526, the stability manager525, and/or the UHM 510, such as timing for predicted collision,instability, or user health issues, and coordinate one or more alertswith the time for the predicted collision, instability, or user healthissue for crash mitigation or condition avoidance. This timing may bebased on expected user reaction times and user health considerations. Insome embodiments, an audio alerting function or 3D audio is implemented.When an obstacle or dangerous condition is detected, one or more soundsare produced from the direction of the threat by one or more speakersinstalled on or around an S-MMS 18.

In one embodiment, the necessary speakers are integrated as part of theHMI and controlled by the HMI processor 208 (FIG. 2). Other embodimentsmay alternatively or additionally include visual, haptic, or biofeedbackmechanisms to provide a corresponding feedback.

The alert manager 540 may also alert the user to degraded systems thatrequire maintenance to keep systems operational. One example of this isintegration of LED lights on each sensor or associated with each sensorprocessor 214 (FIG. 2). In this example, the alert manager 540 monitorsthe status of the sensors (e.g. by receiving a status signal from eachsensor) and transmits a signal to each sensor LED when that sensor failscausing the LED to turn on and notify users and technicians of a failedsensor. In one example, the sensor status signal is either an on state(positive voltage signal) or an off state (no voltage). The alertmanager 540 may receive multiple types of status signals from thesensors and/or other devices of the S-MMS 18 and transmit one or moresignals to one or more status indicators (e.g. single or multi coloredLED or other visual, audio, or signal indicator) to cause to indicatorto alert the user, technician, or other person as to the status of thesensor or other device. These “Clean Lights” or other indicators mayalso be clustered in a central display as part of the HMI 352.

FIG. 6 depicts a front-wheel drive S-MMS 18B equipped with an embodimentof sensor arrays to cover three main types of sensor coverage zones,including zones 610 (zones with solid lines), zones 620 (zones withdashed lines), and zones 630 (zones with dash-dot lines). The depictedsensor coverage zones 610, 620, 630 are covered using three main sensorarrays, including five forward sensors, five rearward sensors, and sixside sensors (three for each side) generally aimed to cover the range ofmotion of the S-MMS 18B and toward the blind spots of a user. Thesensors may be arranged to take readings on the area proximate to theS-MMS 18. Additionally or alternatively, the sensors may be arranged totake readings on the terrain (e.g. ground) proximate to the S-MMS 18.The depicted sensor arrays are composed of a mix of (a) short-rangesensors such as short-range LIDAR, SONAR, and RADAR sensors for zones610 with solid lines, (b) long-range sensors such as long-range LIDARand RADAR sensors for zones 620 with dashed lines, and (c) imagecapturing sensors (including still images and/or video, referred togenerally as imagers) such as cameras for zones 630 with dash-dot lines.

Generally, short-range sensors have wider fields of view (FOV), whilelong-range sensors have narrower fields of view. The sensor array ofFIG. 6 may be characterized as an overlapping, large FOV (with bothnarrow and wide FOVs), sensor array. The sensor placement, number ofsensors used, and types of reports expected in the depicted embodimentare used as a non-limiting example. It should be clear that more orfewer sensors of varying types may be used.

One or more single sensor units may have the ability to act in bothshort and long-range modes. One or more single sensor units may have theability to measure objects and measure or map the terrain proximate tothe S-MMS 18. One or more sensors and sensor types may be selected for adesired S-MMS 18 architecture and user. As an example, if the user is aquadriplegic and spends most of their time indoors in a relativelyslow-moving S-MMS, the sensor arrays may be comprised of all short-rangesensors, with multiple overlapping FOVs, where the time to impact on aclose-in maneuvering object is much more critical than a vehicle on thehorizon. Alternatively or additionally, the sensor array would beadjusted if fitted to alternative drive configurations such as a centeror rear-wheel drive S-MMS.

When working with an overlapping, large FOV, sensor array, MultipleObject Tracking (MOT), or Multiple Target Tracking (MTT), techniques maybe used. For example, a threat assessor (TA) function 528 (FIG. 5) ofthe SAC 302B is responsible for locating multiple objects, maintainingtheir identities, and calculating their individual trajectories givensensor input data (e.g. sensor reports 372-374). Objects to track canbe, for example, pedestrians on the street, vehicles in the road, sportplayers on the court, obstacles or features of the ground, surfaceconditions of upcoming terrain, or animals. Further, multiple “objects”could also be viewed as different parts of a single object. This is thesituational awareness map or tactical awareness map (e.g. a map and/oran identification of the ground, conditions, surfaces, and/or objectssurrounding the S-MMS) maintained by the tactical manager 527.

FIG. 7 depicts an example of 360-degree situational awareness. Multipleobjects may be present at any given time, arranged around the S-MMS 18Bat varying distances from the user. These objects may include featuressuch as a curb or drop 751 in the ground that could be dangerous tostability, moving objects such as a person 752 or bicycle 753, andstationary objects such as a lamp post 754. The S-MMS 18B may becomeaware of objects in the distance 720, yet not be able to confidentlyidentify the objects until they are in range or more information can begathered. For instance, the S-MMS 18B may not be able to confidentlydifferentiate a bicycle 753 from a motorcycle when it is a long distance720 away. Objects in midrange 730 may be more easily recognizable, andobject identity may be qualified and quantified. “Quantify” refers tothe determination of aspects such as range, bearing, location, velocity,and acceleration, i.e. the kinematic state of an object. In closeproximity 740, the SAC 302 (FIG. 3) may generate threat/collisiondata/information. “Close” may be assessed in terms of time to impact,distance, and/or general threat.

In practice, the zones depicted in FIG. 7 extend 360-degrees around theS-MMS 18B. Each side may have varying degrees of need for the size andplacement of the zones dependent on several factors, including S-MMSdynamics, which in turn may dictate the number and type of sensorsrequired, in some embodiments. The depicted zones are for examplepurposes only and may vary between embodiments depending on sensortypes, number of sensors, sensor ranges, characterization data,historical data, and other related systems, devices, andinformation/data.

Uniform Cartesian Grid

The SAC 302B, in some embodiments, is a tactical manager that operatesin a uniform way at a systems level with all of the sensors, data, time,etc. coordinated into a unified system. Some difficulties in uniformoperations include that target reports from the sensors are typically inpolar coordinates (e.g. range and bearing) to a target from the centerof the sensor aperture or receptor, that the sensors are mounted atdistributed points on and around the S-MMS 18 to achieve the best fieldsof view, and that processing time, sweep rate, communications time, andother sensor operating parameters may vary with each sensor. The framesof reference for each sensor can be identified and described in terms oftheir relationship to each other and their functional roles. Thesedifficulties may be overcome using the following methods.

There are five frames of reference in an S-MMS 18 sensor system, wherethe S-MMS 18 is designed to navigate the real world. These frames ofreference are:

-   -   1. Sensor Frame.    -   2. MMS Frame.    -   3. Navigation Frame.    -   4. Earth Frame.    -   5. Geo Frame.

A sensor frame extends from the center of an aperture or receptor (e.g.lens, antenna, etc.) of the sensor. FIG. 8 depicts typical sensors 802,804. In the depicted embodiment, the x-axis of each sensor extendsorthogonal to the direction of travel and indicates roll attitude, they-axis of the sensor extends in the direction of travel and indicatesthe pitch axis, and the z-axis of the sensor extends vertically throughthe center of the sensor and indicates the yaw.

FIG. 9 depicts an exemplary mobile chair S-MMS 18C frame of reference.In the depicted embodiment, the x-axis extends from left to right of themobile chair S-MMS 18C and is parallel to the front of the mobile chairS-MMS, the y-axis is perpendicular to the x-axis extending forward inthe direction of travel, and the z-axis is orthogonal to the x andy-axis and vertical through the intersection of the x and y axes. Aswith the sensor reference frames, x, y, and z-axes represent roll,pitch, and yaw measurements.

In some embodiments, an S-MMS 18C may have multiple frames of reference,one of which may be considered the master frame of reference. As anillustrative example, the S-MMS 18C may have suspended assemblies (suchas wheel cowlings) that move relative to the S-MMS frame itself. If asensor were mounted on a suspended assembly, then the motion of thesuspension would need to be accounted for when translating a sensorframe of reference to the master frame of reference.

FIGS. 10A-10C depict an exemplary navigation frame of reference overlaidon a mobile chair S-MMS 18C. The navigation frame of reference relatesto how an S-MMS 18C is traveling in its environment. FIGS. 10A and 10Bdepict an exemplary navigation frame of reference. FIG. 10C depicts asituation in which the actual navigation frame of reference and thesensor frame of reference do not align due to error. In this scenario,the S-MMS 18C sees the world as x′ and y′-axes, but in reality, theS-MMS travels in x and y-axes. In this example, x and y-axes are theactual navigation frame, regardless of what the sensors detect. As ananalogy, FIG. 11 depicts an airplane 1110 traveling through the air. Thepilot of the airplane 1110 thinks it is traveling along a y-path when inreality it is traveling along a y′-path. Regardless of the sensorreports along the y-path, the plane 1110 is traveling along the y′-path.If an object like a small plane 1120 is in the true navigation frame ofthe plane 1110, there will be a collision. An understanding of thisprinciple is important to safety and object avoidance. The reported datafrom sensors reporting along a y-axis of an S-MMS 18C frame, and/orother sensor frames, will be different in the navigation frame asdescribed. The SAC 302B tactical manager 527 properly associates objectsand conditions reported by sensors (371-376) with the true navigationframe of the S-MMS 18C. Below, further systems and methods are disclosedfor detecting and managing these frames.

The next two frames of reference are the earth frame and geo frame. Theearth frame is in terms of latitude, longitude, and altitude asrepresented on a map. The geo frame is earth centered and earth fixed interms of direction cosines. The intersection of these curved lines fromthe center of the earth represent where an object is located threedimensionally above or below the surface of the earth, as when travelingin an airplane or submarine. These two terms are sometimes referred toas the geoid and spheroid. Use of the geo frame and earth frame may beuseful for navigating between floors on a multi-story building.

Altitude changes of the surface of the earth frame (e.g. terrain) areparticularly important for safe and effective navigation of S-MMSs toavoid tipping and/or instability. The need of an S-MMS 18C to map andcharacterize terrain at a high accuracy is unique to S-MMSs compared toother types of vehicles.

Establishing the UCG

With focus on the first four frames of reference, a uniform way ofreceiving, using, and sharing information needs to be established.Sensor reports (372-375, FIG. 3) and external data (371, FIG. 3) may bereceived by the situational awareness controller 302B relative todifferent frames of reference. A method for managing this is toestablish a uniform Cartesian coordinate system extending from the S-MMS18C reference frame depicted in FIG. 9. All data received by the SAC302B may be translated to the S-MMS 18C reference frame, where the earthreference frame is also referenced to the S-MMS 18C reference frame bynavigation 363. Sensor reports received in polar coordinates (i.e. rangeand bearing to objects) are first translated into Cartesian coordinatesby the SAC 302B through what is called a standard conversion, disclosedbelow. Then, those converted sensor reports may be translated by the SAC302B into the S-MMS 18C reference frame and related to the earth frame.

Motorized mobile systems enable their users to navigate the world. Theynavigate and move in a distinct way, at a different speed, and occupy avery different footprint from the people or animals walking around them.This unique behavior, combined with the power and weight of the MMS,make awareness and avoidance of people, animals, and other things thatmove important to the safety of both the MMS user and those movingaround it. As opposed to simply avoiding collisions like an autonomousvehicle, these systems need to watch for and avoid toes and tails. S-MMS18 collision avoidance and navigation systems must be much more accuratethan existing automobile systems and must take into consideration uniquesituations.

The uniform Cartesian grid (UCG) lays the foundation for creating avirtual situational awareness (SA), or tactical, map. The situationalawareness map comprises objects, the kinematic states of these objects,and state propagation with respect to the user's S-MMS 18 and theobjects in their environment. The threat map is constantly updating withdata reported by onboard sensors (see FIG. 6), data reported by remotesensors via CNI 371, and data received via the internet or otherexternal sources using one or more onboard communication processors 216,FIG. 2. The SA map may be maintained by the tactical manager 527 of theSAC 302B and allows the S-MMS 18 to calculate time to impact onproximate objects. The UCG becomes the single corrected frame ofreference for all calculations on the SAC 302.

The S-MMS 18, as disclosed, uses sensors capable of making measurementswith the lowest possible amount of error, or variance. Variance may bedefined in terms of plus or minus in seconds, milliseconds, feet,inches, yards, degrees, radians, etc. Sensor measurements are used insupport of a logic based S-MMS controller 110B system that may beinferenced by some form of adaptive learning that operates likeintuition, as discussed further below. Two or more sensor reports on thesame object that are different and can be associated in time may becombined to generate a better report, with the goal of improvingconfidence in the tactical map.

Referring back to FIG. 6, object 640, is covered by four sensors: twoshort-range LIDAR as designated by zone 610, one forward imager asdesignated by zone 630 and one long-range radar as designated by zone620. A process is necessary for reconciling these multiple sensorreports (372, 373, and 374) on the single object 640 received by the SAC302B. Preferably, this method would materially improve confidence in theaccuracy of the system tactical map maintained by the tactical manager527, thus improving situational awareness. As discussed above, the SAC302B may implement a probability based system with an estimator.

Dual Mode Variance

Dual mode variance is a method for improving confidence in thesituational awareness map of the SAC 302. Dual-mode variance may be usedto decrease uncertainty of any S-MMS 18 functions disclosed wherein twoor more sensor reports (371-376), or readings from multiple modes on asingle sensor, can be combined to decrease uncertainty. An approach isdepicted in FIG. 12 and comprises a first object 1202, located proximateto a motorized mobile system 18, wherein the first object 1202 is in thefield of view 1204, 1206 of at least two sensors 1208, 1210 associatedwith the motorized mobile system 18. In some embodiments, a first sensor1208 generates data about the first object 1202, wherein the firstobject data is represented by one or more of a first range measurementand a first bearing measurement to the first object. A range, or downrange, measurement is the distance of something to be located from somepoint of operation, typically defined by the sensor frame of reference.A cross range or bearing measurement is a direction expressed in degreesoffset from an arbitrary true baseline direction, typically defined bythe sensor frame of reference. The first range measurement may have anassociated first uncertainty with the first range measurement, andwherein the first bearing measurement has an associated seconduncertainty with the first bearing measurement. As a non-limitingexample, the first sensor 1208, an imager for example, may provide goodcross range or bearing measurements but poor down range measurementssuch that the first uncertainty is larger than the second uncertainty asdepicted by zone 1212. A second sensor 1210 may generate data about thefirst object 1202, wherein the first object data is represented by oneor more of a second range measurement and a second bearing measurementto the first object. The second sensor 1210, a radar for example, mayprovide good down range measurements but poor cross range measurementssuch that the second range measurement has an associated thirduncertainty with the second range measurement, and wherein the secondbearing measurement has an associated fourth uncertainty with the secondbearing measurement, wherein the fourth uncertainty is larger than thethird uncertainty as depicted by zone 1214.

These overlapping sensor reports may be coordinated in time andassociated with a single object (a.k.a. track) by sensor track fusion520 on the SAC 302B. The most accurate measurement from each sensorreport (cross range from the imager and down range from the radar, inthe example) may be combined to make the situational awareness map moreaccurate. This use of more than one sensor is referred to as dual modevariance. In order to accomplish this, a processor (e.g. S-MMS processor202), may host one or more programs such as the SAC 302B of the S-MMScontroller 110B to:

-   -   receive from the first sensor data about the first object 1202,    -   receive from the second sensor data about the first object 1202,    -   retrieve from memory 120 the range and bearing uncertainties        associated with the data generated from the first sensor,    -   retrieve from memory 120 the range and bearing uncertainties        associated with the data generated from the second sensor,    -   identify the lower bearing uncertainty from the second and        fourth bearing uncertainty,    -   identify the lower range uncertainty from the first and third        range uncertainty,    -   combine the measurements associated with the lower bearing        uncertainty and lower range uncertainty to generate a new        reduced area of uncertainty (e.g. 1216 as compared to 1218),        wherein uncertainty refers to potential errors in physical        measurements that are already made.

As an example, FIG. 12 depicts an embodiment comprising two sensors1208, 1210 operating with different fields of view. Imager 1210 has afield of view of 120-degrees 1206, and is based on a 1920×1080resolution camera. This equates to an accuracy of 16 pixels per degree(1920-pixels/120-degrees=16 pixels per degree). If the detectionsoftware (e.g. running on the sensor processor 214) has a cross rangeresolution of ±2 pixels for object detection, there is a cross rangeerror of ±0.125-degrees on the image sensor reports 374 from the imager1210 received by the SAC 302B. The down range error for the imager 1210is reported by the camera manufacturer as half the distance to thetarget. If the detection software on the imager 1210 estimated thetarget at 100-meters, the down range error is ±50-meters. This resultsin a variance, or error envelope, that looks like 1212. If the secondsensor is a radar 1208 with an associated field of view 1204 of30-degrees and that is reported to have down range errors in the ±10-cmrange, but cross range errors of ±3-degrees its error envelope isdepicted as 1214.

If, for example, these two sensor reports 374 and 372 are received bythe SAC 302B, associated with a single object (a.k.a. track) by sensortrack fusion 520, and unchanging over the last 30-seconds, theconfidence is high that both sensors 1208, 1210 are targeting the sameobject. In one embodiment, the tactical manager 527 places the reportson the situational awareness map, and the object's location is marked assomewhere in the field of uncertainty depicted by 1218. Alternatively,if dual mode variance is employed, the best (e.g. lowest variance)elements of each sensor 1208, 1210 readings are combined, and theobject's location is marked as somewhere in the field of uncertaintydepicted by 1216. This is a significant reduction in uncertainty ofobject parameters. Without application of the disclosed approach, theuncertainty of object parameters would be the combination of the maximumerrors or variances 1218. This improvement is termed dual-mode variance,and any single sensor reports are termed single-mode variance (e.g. 1212or 1214).

As another non-limiting example, a single radar using a dual-modesynthetic aperture may be used, where the fields of view have twooperating modes, a short range wide mode and a long-range narrow mode.The sensor report in short range wide mode may have similarcharacteristics to the imager 1210 discussed in the first example (i.e.good cross range accuracy, but poor down range accuracy). The sensorreport in long range narrow mode may have similar characteristics to thetraditional radar 1208 discussed in the first example (i.e. poor crossrange accuracy, but good down range accuracy). If two measurements aretaken from the single dual-mode synthetic aperture sensor, one in eachmode, the measurements can be combined by the SAC 302B using dual-modevariance to increase confidence in the properties of a given object.

The previous examples are presented for clarity and are not intended tolimit the types of objects, sensors, or scenarios where dual modevariance may be utilized. The first and second sensors may be one ormore of an optical sensor, a sound sensor, a hall effect sensor, aproximity sensor, and a time of flight sensor, including radar, sonar,ultrasonic, LIDAR, or a distance camera. An object may include one ormore of a thing, person, animal, ground feature, and surface condition.Additionally or alternatively, applications may include combinations ofbio-medical sensors to track user health 375, ground or terrain trackingsensors to monitor system stability 372-374, and/or S-MMS status sensorsincluded as part of the motor controller 351 or HMI 352, among others.Additionally or alternatively, one or more of the sensors generatingdata about an object may be received from a second S-MMS 18 or remotesensor associated with the S-MMS 18.

Polar to Cartesian Conversion

Another method of improving situational awareness and increasingconfidence in the situational awareness map is application of polar toCartesian conversion techniques. An inertial measurement unit (IMU) maybe attached to the frame of an S-MMS 18 so that it reports the absoluteorientation, attitude, and heading of the S-MMS 18 to the S-MMScontroller 110B via the GPS and inertial manager 376. These reports maythen be used to support the navigation 363 function of the S-MMScontroller 110B. Additionally, the IMU sensor reports may also be usedby the S-MMS controller 110B and SAC 302B to establish a vehiclereference frame extending along the S-MMS axes, as illustrated in FIG.9.

Having a vehicle reference frame is necessary in the move to smartsensors that generate target sensor reports in a polar coordinatesystem, where the sensor reports are not in terms of x, y, and z, but interms of range and bearing. Moreover, these sensor reports are receivedbased on one or more sensor reference frames (FIG. 8). Navigation 363(FIG. 3) is responsible for maintaining the vehicle reference frame. TheSAC 302B then translates data from sensor reports representing targetsin the environment and other data into a uniform Cartesian coordinatesystem allowing a higher confidence in the determination of threats.

As the sensor system evolves from single sensor reports to an integratedsystem of sensors, new issues arise, such as the capture and managementof system error. In the example of axis translation as sited above, thepolar translation may be completed in a standard conversion, asdescribed in Equation 1:x _(m) =r _(m) cos θ_(m) and y _(m) =r _(m) sin θ_(m),  Eq. 1where r_(m) and θ_(m) are the range and bearing, respectively, of thesensor target in the polar reference frame, and x_(m) and y_(m) are thedown range and cross range coordinates, respectively, in the convertedCartesian reference frame. As an example, for a target at 10-meters,with a bearing of 1-degree, Equation 1 yields a Cartesian location of(9.99-meters, 0.17-meter) for (x,y). This conversion leads to someerror. However, for the scenarios disclosed, the errors induced will benegligible. Additionally or alternatively, the SAC 302B may de-bias thevariances of the polar reports when they are translated to Cartesiancoordinates. A debiasing term may be used in the case that the standardEquation 1 cannot be used because they induce significant error. FIG. 13depicts the standard conversion as well as the use of a debiasedconversion correction term.Cooperative Targeting

Cooperative targeting is another way of improving the situationalawareness of the S-MMS 18. As stated earlier, the communication,navigation, identification (CNI) 371 (FIG. 3) function of thearbitration IAM 370 manages off-board communication for the S-MMS 18through system links and off-board links to enable S-MMS 18coordination. CNI 371 is also responsible for system wide time andprocessor time synchronization across the system in conjunction with theoperating system of the S-MMS controller 110B.

Communication standards have been developed to support and enhanceautomobile communications known as vehicle-to-vehicle (V2V) orvehicle-to-infrastructure (V2I). The standards define an architectureand a complementary set of services and interfaces that collectivelyenable V2V and V2I wireless communications. Together, these standardsprovide the foundation for a broad range of applications in theautomobile transportation environment, including automobile safety,automated tolling, enhanced navigation, traffic management, and manyothers. One standard was adopted as IEEE 1609 for Wireless Access inVehicular Environments (WAVE) in the United States. The EuropeanTelecommunications Standards Institute (ETSI) Technical CommitteeIntelligent Transport System (ITS) developed and adopted a related setof standards, collectively called the ITS-G5. The WAVE and ITS-G5standards are herein incorporated by reference in their entirety.

V2X communication is based on WLAN technology and works directly betweenvehicles or infrastructure, which form a vehicular ad-hoc network, astwo V2X transceivers come within each other's range. Hence, V2Xcommunication does not require any infrastructure to communicate, whichhelps to increase safety in remote or little developed areas. V2X isparticularly well-suited for S-MMS 18 communication, due to its lowlatency and the ability to communicate instantly with no surroundinginfrastructure. V2X transmits messages known as Common AwarenessMessages (CAM), Decentralized Notification Messages (DENM), or BasicSafety Messages (BSM). Messages may contain a range of content,including the identity, type, and kinematic states of the entitycontaining each V2X transceiver.

FIG. 14 depicts two mobile chair S-MMSs 18D and 18E in communicationwith each other via an ad hoc network 1404. In an embodiment, the twoS-MMS 18D and 18E may be configured to communicate with each other overa V2X network connection. The S-MMS 18D, 18E may be equipped with an802.11P transceiver of the S-MMS 18 in communication with thecommunications processor 216 and the CNI 371 configured to manage802.11P communications. In this way, the two S-MMSs 18D, 18E form acommunication system comprising a vehicular ad hoc network forexchanging messages. The vehicular ad hoc system comprises a firstsecurity manager located in a portable first transceiver, wherein theportable first transceiver is one or more of proximate to the S-MMS 18Dand physically coupled to the S-MMS 18D, linking data for transmissionoff-board to a second processor and second security manager associatedwith a second S-MMS 18E connected to the vehicular ad hoc network.

As a non-limiting example of improving situational awareness usingcooperative targeting, FIG. 14 depicts a first S-MMS 18D and a secondS-MMS 18E approaching a blind intersection of two pathways. The onboardsensors 1402 of the first S-MMS 18D do not detect the second S-MMS 18E,nor do the onboard sensors of the second S-MMS detect the first S-MMS.However, both S-MMSs 18D, 18E are equipped with an 802.11p architecture(including software and hardware as described above, including an802.11P transceiver, to effect the 802.11p architecture), transmit theirkinematic states, and receive signals from the other S-MMS containinginformation on the kinematic states of the other S-MMS via a vehicularad hoc network 1404. In this example, the 802.11P transceiver on thefirst S-MMS 18D picks up the signals from the second S-MMS 18E. The CNI371 of the first S-MMS 18D receives the signals from the second S-MMS18E via the 802.11P transceiver of the first S-MMS and transmits thecontents of the received signals (including the kinematic stateinformation therein) to the S-MMS controller 110B of the first S-MMS viathe arbitration IAM 370. Additionally or alternatively, the signalcontents may be stored in one or more onboard memory locations 120 ofthe first S-MMS 18D for consumption by the S-MMS controller 110B of thefirst S-MMS. The signal contents from the second S-MMS 18E may includethe identification information, kinematic state information (e.g.current location, heading, and velocity of the S-MMS 18E), and estimatederror in the kinematic state information.

The SAC 302B on the S-MMS controller 110B of the first S-MMS 18D may usethe information from the signals received from the second S-MMS 18E inone of many ways. In one embodiment, the SAC 302B of the first S-MMS 18Dmay identify a track for the second S-MMS 18E based on the informationfrom the signals and directly place the newly identified track for thesecond S-MMS on the situational awareness map via the tactical manager527. The threat assessor 528 of the first S-MMS 18D may then assign aperceived threat level to the second S-MMS 18E based on the newlyidentified track or based directly on the information from the signals.Other SAC 302B functions of the first S-MMS 18D may then act on theinformation from the signals of the second S-MMS 18E as if the onboardsensors of the first S-MMS had detected the second S-MMS, when inreality the second S-MMS was not in view. At the same time that theseactions are occurring on first S-MMS 18D, the second S-MMS 18E may alsobe taking these actions. This is referred to as cooperative targeting.

-   -   In some embodiments, data provided by the second S-MMS 18E to        the first S-MMS 18D over the ad-hoc 1404 network may be combined        with sensor reports 1402 received by the first S-MMS 18D in such        a way that the confidence of additional tracks on the        situational awareness map maintained by the tactical manager 527        may be improved. For example, the confidence of a location of a        particular object as determined by the SAC 302B of the first        S-MMS 18D may be enhanced by combining information (e.g. using        dual mode variance for example) from one or more sensor reports        1402 from the first S-MMS 18D with data about the same object        (e.g. track data) provided by one or more sensors onboard the        second S-MMS 18E via one or more signals from the second S-MMS        that include sensor data from the second S-MMS.

In some embodiments, the S-MMS controllers 110B on each S-MMS 18D, 18Emay automatically coordinate the right of way or other behavior whenthey meet (e.g. at the intersection depicted in FIG. 14). Each S-MMS18D, 18E may process one or more received signals and determine amessage should be sent over the ad-hoc network 1404 indicating thedirection the S-MMS is traveling and the anticipated crossing time ofthe two S-MMSs. In each S-MMS 18D, 18E, the CNI 371 receives a messagefrom its S-MMS controller 110B and transmits the message via its 802.11Ptransceiver. Based on this message, the S-MMS controller 110B on one ofthe two S-MMSs may request permission from the S-MMS controller on theother S-MMS for right-of-way. If the request is approved by the S-MMScontroller 110B of the other S-MMS, then the SAC 302B of the other S-MMSdetermines it is to stop and may alert its drive path manager 529 viaits threat assessor 528 to block its forward motion at a specificlocation and/or time. Additionally or alternatively, the two S-MMS 18D,18E may simply coordinate a slight increase or decrease in velocity ordirection that ensures the two S-MMSs 18D, 18E do not collide at theintersection. In other embodiments, a second message will be transmittedby the S-MMSs 18D, 18E (via their respective components described above)when the intersection has been successfully traversed.

Cloud Integration

FIG. 15 depicts an embodiment of an S-MMS system 1506 securely connectedto a remote server 1510. In the depicted embodiment, a wirelessprocessor either onboard the S-MMS 18F or associated with the S-MMS as apaired or connected wireless smart device 1502 is operably coupled to asecond wireless processor (e.g. via a secure wireless connection),wherein the second wireless processor is configured to operate on apublic packet network. The connection 1508 to the public packet networkfrom the S-MMS system 1506 allows access to one or more servers, orremote services, 1510 configured to receive data from a secure memory onthe S-MMS 18F and/or the secure memory on the associated smart device1502 of the S-MMS system 1506. The received data may then be stored in asecure memory on the remote server 1510 wherein the secured, stored datais accessible to pre-authorized systems or pre-authorized persons.

In an embodiment, a pre-authorized system may be a web interface 1542accessed through an internet connected device, wherein a pre-authorizeduser 1544 logs in using security credentials and may view a history ofevents or other data associated with a particular S-MMS user or theS-MMS of that user. In another embodiment, the web interface 1542 may beused to communicate with the S-MMS user or modify or add data to theS-MMS users' unique data file. Data transmitted to the web interface1540 may be delivered via a secure connection, such as a secure socketslayer (SSL) connection.

In one embodiment, the remote server 1510 may be configured to accept,process, store, and complete specific actions based on messages from anS-MMS system 1506. FIG. 15 depicts a remote server 1510 configured toreceive incoming messages from an S-MMS system 1506 via a secureconnection 1508. Messages may be sent from the S-MMS system 1506 when aparticular event takes place (e.g. on a tipping event), at state events(e.g. at startup of the device), and/or at pre-scheduled times (e.g. atmidnight each night). These incoming messages are received by an inputqueue 1512 hosted on the remote server 1510. Messages from the inputqueue may then be sent to one or more of a compute engine 1514, adatabase or data storage system 1516, and an output queue 1518. Thecompute engine 1514 may compare data included in incoming messages fromthe input queue 1512 to predefined rules. Additionally or alternatively,the compute engine 1514 may work as a filter to associate individualmessages with unique user accounts prior to storing data in a database1516. The compute engine 1514 may also, based on the content of areceived message, push alerts or action requests to an output queue1518. The output queue 1518 may be a subscription/publication servicethat, when activated, sends alerts to internet connected devices 1532,such as an S-MMS users' caregiver's smart device. Alerts may be sent astext messages, voice messages, voice calls, or video over a cellularand/or Internet Protocol (IP) network 1530, including the internet.Additionally or alternatively, outbound messages may be sent via thecellular and/or Internet Protocol (IP) network 1530. In someembodiments, the services 1512-1518 on the remote server 1510 may beexecuted on one or many individual server instances that work togetherto complete tasks as a single unit on one or more hardware processors.Additionally or alternatively, multiple other remote serverarchitectures are possible to accomplish the intended functionality. Asa non-limiting example dashed-dot lines have been included to showalternative connections that may be enabled in some embodiments.Additionally, the compute engine 1514 may include more advanced machinelearning logic, such as a neural network, in some embodiments.Diagnostics and other basic server components have been left off of thediagram for the purpose of clarity.

Each of the exemplary embodiments disclosed illustrate how the combinedfeatures of the S-MMS system enable various capabilities for S-MMSs.While each exemplary embodiment is discussed separately it should beclear that any one or more aspect of each exemplary embodiment may becombined with any one or more aspects of one or more of the otherexemplary embodiments.

Precision Navigation

S-MMS navigation requires accuracy and a unique set of technologies.Navigation for automobile use relies primarily on GPS for location. FIG.16A depicts an S-MMS 18G configured with one or more communicationprocessors 216 which provide data to the S-MMS controller 110B based ondata received from Global Positioning System (GPS) signals 1602.However, unlike automobiles, S-MMS users may spend much of their timeindoors and in other locations where GPS does not work. GPS-assisteddevices require signal strength for accuracy. When signals weaken (inareas like downtowns or the interiors of buildings), the accuracy lags,potentially causing inaccuracies in navigation that can have adetrimental impact on those who rely on driver assistance or autonomousoperation with high accuracy. FIG. 16B depicts an S-MMS 18G in an indoorenvironment where GPS signals 1602 are weakened or completely blocked.

Moreover, currently available GPS technologies struggle to provide thepositional accuracy required for S-MMS navigation along sidewalks,paths, and other rights of way which are typically much narrower thanroadways. Indoor navigation of multi-story buildings is anotherchallenge that is not currently addressed by vehicle navigation systemsfor obvious reasons. These issues mean that S-MMS 18 navigation requiresa unique set of technologies and methods.

The S-MMS 18 situational awareness controller 302B with tactical manager527 (FIG. 5), and drive path manager 529 lay the foundation forprecision navigation. The tactical manager 527 maintains a situationalawareness map of the surroundings (e.g. a map and/or an identificationof the ground, conditions, surfaces, and/or objects surrounding theS-MMS) based on data provided to the SAC 302B. The drive path manager529 references the situational awareness map when assisting a user orfor autonomous navigation. The combination of sensors available to theS-MMS controller 110B via the arbitration IAM 370 includes GPS andinertial manager 376, multiple onboard sensors previously disclosed372-374, information available from remote services 1510 and third-partyinformation available via the CNI 371 which, together, may supportcreation of a precise S-MMS situational awareness map at a level ofdetail not otherwise available.

Navigation 363, in an embodiment, may rely at least partially on aninertial reference system (IRS), including an inertial navigation system(INS) for navigating using dead reckoning (DR). Dead reckoning is theprocess of calculating the S-MMS 18G (FIG. 16) current position by usinga previously determined position, or fix, and advancing that positionbased upon known or estimated speeds and/or steering over elapsed timeand heading. In one embodiment, depicted in FIG. 16B, an S-MMS 18Genters a building and loses a GPS signal 1602. The DR function innavigation 363 uses location data from the GPS and inertial manager 376to update the INS DR fix 1604 by setting a stable earth frame ofreference. Speed, heading, and elapsed time data is then provided by GPSand inertial manager 376. Navigation 363 uses this information to trackthe movements 1608 of the S-MMS 18G (or more precisely it's vehicleframe of reference) relative to the fix 1604 and estimate its location.This allows the S-MMS 18G to navigate inside a building with GPS orotherwise (e.g. otherwise outside of GPS coverage) to a similar degreeof accuracy as navigating outside of buildings with continuous GPS dataor otherwise. The approach may also be utilized when GPS coverage isavailable to increase locational accuracy by cross-checking the DR valuewith GPS data.

Navigation assistance and autonomy may be strengthened further andsmoothed by combining DR, as previously disclosed, with the SAC 302Bsituational awareness map maintained by the tactical manager 527 (FIG.5). This map is maintained with information from the collision manager526 and stability manager 525 and their associated sensors. While GPSand DR provide an estimate of absolute location, the situationalawareness map provides information on safe directions of travel anddistances to surrounding objects. For example, when traveling down ahallway, navigation 363 may know the current location and target endlocation of the S-MMS 18G, and the SAC 302B is aware that the S-MMS muststay between the two walls to get to that location. The drive pathmanager 529 of the SAC 302B references the situational awareness mapwhen assisting user or autonomous navigation. This added awarenessallows navigation of tight spaces that would not otherwise be possiblewith DR alone. Increased accuracy of the situational awareness mapimproves navigation capabilities and, therefore, use of dual modevariance improves S-MMS 18G precision navigation capabilities.

With reference to FIG. 16, in an embodiment, the CNI function 371 (FIG.5) is provisioned such that at least one wireless system is available totransmit and receive (transceive) data relevant to navigation of anS-MMS 18G. In some embodiments, the S-MMS 18G may be compatible withcellular, Bluetooth, Bluetooth Low Power, Wi-Fi, and other signals whichcan be used for navigation assistance. These signals may be used inaddition to, or as an alternative to, GPS signals. Additionally oralternatively, the S-MMS 18G may use Bluetooth, Bluetooth Low Power,Wi-Fi, and other signals, such as light frequency navigation as sensedby a paired smart device, for instance, being deployed for indoortracking and navigation by several companies. In one embodiment, awireless beacon 1610, communicates 1612 with the S-MMS 18G via acommunication processor 216 (FIG. 2) of the S-MMS 18G through one of802.11, Bluetooth, Bluetooth Low Energy (BLE), Near Field Communication(NFC), and/or Zigbee™ among others. These communications 1612 aretypically predicated on a “just works” model, one that requires littleinterface with the device 1610, 18G and the underlying technology. Asbeacon and indoor navigation systems grow in use, S-MMS 18G precisionnavigation may take advantage of them to more accurately determine thelocation of the S-MMS 18G with respect to the earth reference frame.Additionally or alternatively, the S-MMS 18G SAC 302B may use thisinformation to improve mapping of objects and conditions on thesituational awareness map as maintained by the tactical manager 527.

As previously disclosed (FIG. 14), two S-MMSs 18G, 18H may communicatewith each other via an ad hoc network. In an embodiment, the S-MMSs 18G,18H may be equipped with an 802.11P transceiver in communication withtheir respective communications processors 216, and their CNIs 371 areconfigured to manage 802.11P communications. In this way, the two S-MMSs18G, 18H form a communication system comprising an ad hoc network forexchanging messages. The contents of messages transmitted from the twoS-MMSs 18G, 18H may include information relevant to navigation. In oneimplementation, the messages may contain information on the currentlocation and kinematic state of the respective S-MMS 18G, 18H.Additionally or alternatively, the messages transmitted from the twoS-MMS 18G, 18H may share location and kinematic state of tracks eachS-MMS is tracking. These tracks may then be translated by the receivingS-MMS 18G, 18H from a vehicle frame of reference to an earth frame ofreference and added to or combined with existing tracks on the SAC 302Bsituational awareness map maintained by the tactical manager 527. Inthis way, two S-MMSs 18G, 18H can benefit from enhanced situationalawareness though cooperative targeting of objects, conditions, andevents proximate to each.

In some embodiments, the S-MMS 18G may be equipped with one or moresensors intended to determine the altitude (earth reference frame) orone or more earth centered direction cosines (geoid reference frame).These sensors may include an onboard barometer or altimeter. In oneembodiment, an onboard altimeter reports earth-referenced altitude tothe SAC 302B via GPS and inertial manager 376 and/or navigation 363.This information may allow the S-MMS 18G to determine its location in amultistory building for improved navigation and coordinate with theS-MMS 18H by transmitting that information to the s-MMS 18H in one ormore messages, thereby working cooperatively in the same multi-storybuilding. Additionally or alternatively, altitude data may be receivedby an S-MMS 18G or 18H from one or more messages transmitted from apaired smart device 1502 and/or off board beacon 1610, in either casebased on sensor readings from one or more sensors on the smart deviceand/or off board beacon that are included in the one or more transmittedmessages.

Best Route

Precision navigation capabilities are strengthened when combined withthe availability of high quality maps. Moving through new environmentsin an MMS can often be very challenging. Often ramps and elevators areout of the way, and if MMS users don't know which direction to go, theymay end up getting lost or having to backtrack. For power chair MMSsystems, oftentimes the handicap parking space is located across theparking lot from an accessible ramp. New buildings, let alone newcities, may be a daunting challenge in the unknown for an MMS user. Lackof familiarity makes navigation in new areas difficult, and often isvery time-consuming. Without pre-existing knowledge of a route andaccessibility in an area, an MMS user could easily be overwhelmed andultimately lost or stranded. When an MMS user is moving through anoutdoor environment, something as common as a gap in the sidewalk may bea complete roadblock, as some MMSs may tip if they encounter drop-offsas little as two-inches. Currently available maps do not provide thelevel of detail or information required for MMS users to plot accessibleroutes.

The same sensors and advanced navigation and safety systems in the S-MMS18 that allow it to avoid dangerous drop-offs and collisions may beadditionally used to capture valuable data regarding accessibility,terrain, and preferred paths as users navigate through the environment.This data may be used both to create maps of areas that have not yetbeen mapped with the level of detail required for MMS users and toupdate areas that have already been mapped. Sensor data on sidewalks,interiors of public spaces, and parking lot hazards may be collected andstored by the S-MMS 18 and associated with one or more maps. Theincreased accuracy of dual mode variance readings, as previouslydisclosed (FIG. 12), may aid in producing high quality maps. In someembodiments, the combination of camera data (e.g. via image sensorreports 374) and distance data (e.g. via non-contact sensor and S&Tsensor reports 372 or 373) may allow the SAC 302B to create extremelydetailed 3D digital re-creations of locations and features based on theS-MMS 18 situational awareness map. The completeness and detail of thesemaps may vary based on the sensors available on the S-MMS 18, the pathof the S-MMS through the location, the amount of processing powerassigned to the SAC 302B, and the amount of memory 120 partitioned foruse by the tactical manager 527. Additionally or alternatively, aseparate mapping function may be added to the S-MMS controller 110B tomanage mapping of locations. This may be incorporated as part of the SAC302B or separate from the SAC.

In some embodiments, data associated with one or more S-MMS 18 sensorsand part or all of the situational awareness map maintained by the SAC302B may be shared with remote services and servers 1510 (FIG. 15).Additionally or alternatively, the data may be shared with a pairedsmart device 1502 as previously disclosed.

FIG. 17 depicts an embodiment of an S-MMS 18I wirelessly connected witha remote server 1510B. The remote server 1510B may be configured toaccept information related to mapping and navigation from the S-MMS 18Iover a public packet network, such as the internet. Additionally oralternatively, the information related to mapping and navigation may beseparated by a rules engine 1706 so that any personally identifiableinformation is stored as private data 1702, and non-identifiableinformation is added to or merged with public mapping data 1704. In anembodiment, received data may be further anonymized using published bestpractices for data anonymization by scrubbing data based on predefinedrules 1708 or a program that modifies or mixes information (1514, FIG.15) prior to being placed in public data 1704. In an alternativeembodiment, data may be sorted and/or anonymized onboard the S-MMS 18Iprior to being transmitted to the remote server 1510B. In someembodiments, public data relevant to navigation can be overlaid on, orincluded in, existing map programs, such as Google® maps or WAZE™,adding significantly to the resolution of the data available for a givenlocation. In this embodiment, the S-MMS 18I may connect directly to aservice over a public packet network, or public data 1704 from a remoteserver 1510B may be fed to a service or location for use by a 3^(rd)party.

As illustrated in FIG. 18, as more S-MMSs 18I-X navigate an area andupload their data, map(s) 1802 become more accurate and stay up to date.In some embodiments, users, S-MMSs 18I, 18X, and others may insert pins1804, much like on the current WAZE™ application, to alert others of newconstruction and other potential hazards that may affect accessibilityof MMS users in an area (i.e. broken power-assist doors or elevators,cracks in the sidewalk, closed ramps, closed trails etc.). In someembodiments, the S-MMSs 18I, 18X may add such pins 1804 to a mapautomatically by detecting areas that S-MMS users must consistentlyavoid, areas that S-MMS users must consistently travel through, and/orby using feature recognition. In one embodiment, a machine learningengine 1710 on the remote server 1510B by be configured to recognize andmark these features based on public data 1704 compiled from many users.Rules 1708, such as a confidence level, may be established and enforcedby a rules engine 1706 before public data 1704 is modified to includethese new features. Whether an S-MMS 18I automatically uploads map dataand/or adds pins to map data (including uploaded data) or requires userinput may be dependent on user preferences and/or settings. In someembodiments, an S-MMS 18I may automatically and upload (optionallyanonymously) map data to a remote server or service with or withoutadded pins or alternately only upload pinned locations to the remoteserver or service.

Smart MMSs 18I may allow a user to find the most accessible route totheir destination based on a combination of the onboard situationalawareness map maintained by the situational awareness controller (302B,FIG. 5), paired sensors or beacons 1610 (FIG. 16B), and cloud based mapdata 1802 (FIG. 18) as previously disclosed. In some embodiments, mapdata 1802 from one or more sources may be loaded in the S-MMS 18 systemmemory 120 and/or into the memory of a paired device 1502 such that auser may navigate an area using local map data even when they have nointernet connection. In some embodiments, the S-MMS 18I may utilize mapinformation 1802 as an input to the drive path manager 529 (FIG. 5) ofthe SAC 302B to automatically direct an S-MMS 18 user to a locationusing the most accessible route. In some embodiments, the best route maybe determined by the SAC 302B (by the drive path manager 529 orotherwise) using one or more of time, accessibility confidence,elevation change, and safety as weighting factors. In some embodiments,users may adjust preferences and/or settings affecting how a best routemay be determined. In some embodiments, an S-MMS may be capable oflearning from trial and error and/or user input/data to shift weightingfactors to provide better best route suggestions. The machine learningor rules engine required for this task may reside on the S-MMScontroller 110B or may be hosted on a remote server 1510.

S-MMS Mapping

Whether an S-MMS 18 is connected to the cloud or not, it is capable ofgathering data relevant to mapping and navigation. Many of the disclosedsensors used to avoid collisions and manage the stability of the S-MMS18 (e.g. via reports 372-376, FIG. 3) create detailed profiles of thesurroundings and ground around an S-MMS 18. In some embodiments, othersensors may be incorporated to specifically improve mapping dataacquisition. If the remote server 1510 is offline, mapping data may besaved to secure local memory 120 until such time as the S-MMS 18reestablishes a network connection 1508 with the remote server. Whenmapping areas offline, the data gathered by the S-MMS 18 may be mappedwith respect to the nearest or most recently known location to improveaccuracy when the information is uploaded. While the mapping system isdescribed as essentially cloud-based in some embodiments, it should beclear that the same principles may be applied to mapping applicationsthat may never, or rarely, share information via the Internet.

S-MMSs 18 equipped to continuously measure and evaluate theirsurroundings can create highly accurate maps of the accessibility ofeach location they traverse, as illustrated in FIG. 19. As multipleS-MMSs 18I-K upload information, often mapped with respect to differentknown locations, a remote mapping system (e.g. hosted on a remote server1510) is able to receive the multiple streams of data (e.g. 1512),overlay and fuse the information to eliminate outliers (e.g. via acomputation engine 1514 and/or rules engine 1706), and create a unifiedpublicly available map 1902, stored in a public data repository 1704,for use by S-MMS systems 1506, general caregivers 1544, or smart devices1532. The map 1802B then provides accurate up-to-date navigationinformation to users using the S-MMS 18 accurate frame of referencetracking previously disclosed.

As a non-limiting example, mobile chair S-MMS 18 users navigate theworld in a way that requires a wholly unique set of accommodations whichcan vary greatly between locations. In addition, each accommodation islikely used in sequence (e.g. from an accessible parking spot, to asidewalk ramp, to a different ramp, to a power-assisted door). As mobilechair S-MMS users navigate the world, each link in each sequence is apotential roadblock, unknown to them until they or an assistant arrivesat the next step. Detailed, continuous mapping and logging of theseaccommodations is important to relieving this uncertainty. In someembodiments, S-MMSs 18I-K may be configured to automatically detect andcatalogue any accessibility features 1904, 1906, and 1908 encounteredwhile they are traveling. Automatic detection of accessibility featuresmay be accomplished using one or more techniques currently used forfacial recognition, including geometric, photometric, or 3D recognition.In some embodiments, feature extraction 530 on the SAC 302B may utilizealgorithms, including principal component analysis using eigenfaces,linear discriminant analysis, elastic bunch graph matching using theFisherface algorithm, and the hidden Markov model to identifyaccessibility features. In an embodiment, sensor track fusion 500 maycombine feature extraction 530 outputs with information from one or morenon-contact sensor reports 372 and S&T sensor reports 373 for 3Drecognition of accessibility features. In an embodiment, one or moremachine learning engines (e.g. 1710, FIG. 17) may be used to identifyaccessibility features using published approaches, such as multilinearsubspace learning or dynamic link matching. The totality of thesemeasurements from S-MMSs 18I-K, when combined as previously disclosed,creates a detailed map of not only the accessibility of a specificlocation, but of the sequence of locations through which a user mustpass to their destination, which may be used by other S-MMSs 18I-Knavigating the same location. As an analogy, the sequence of theseevents to enter a building may be equivalent to road informationprovided for automobile navigation. In some embodiments, other systemsand devices, such as wearable devices (wearables) or smart deviceapplications, either associated with an S-MMS 18 or not, may contributeinformation to mapping 1802B by transmitting their data to one or moreS-MMSs or the remote server 1510, and processed and/or stored asdescribed above, and/or may host the mapping application(s).

Indoor locations are a challenge to accurately map because they oftenchange, are often densely populated with objects, and may requirerepeated scans in order to be mapped and kept up-to-date. S-MMSs 18equipped to continuously sense and document their surroundings, aspreviously disclosed, can overcome these issues to create highlyaccurate and regularly updated maps of indoor locations. Not only canthese maps be used to enable accessible S-MMS 18 indoor navigation, theycan also be used by services like Apple's MapKit™ to create veryaccurate indoor navigation and wayfinding for everyone.

In some embodiments, this mapping process may also be linked to 3Dmodels of common objects such as chairs, tables, and doors that can beplaced and associated on a map by either an S-MMS 18 user, a thirdparty, or in an automated fashion to further refine indoor maps. Objectsmay be automatically created based on S-MMS 18 sensor data, custommodeled using a tool such as SketchUp™, or they may be retrieved frommodel databases such as GrabCAD™ FIG. 20 depicts an embodiment whereinthe S-MMS controller 110B of S-MMS 18I uses one or more sensor reports(e.g. sensor reports 373 from a LIDAR and sensor reports 374 from acamera) to sense an object 2050. The object 2050 is assigned a track bythe SAC 302B, and may be characterized as a chair by sensor track fusion500 or by a machine learning engine 1710 (either onboard the S-MMS 18Ior on a remote server 1510B) configured for object detection. Based onthe characterization, the S-MMS 18I then downloads a 3D chair model froma remote service or server 1510C. The tactical manager may thenassociate this new data with the chair track, scale 2030 the 3D chairmodel to match the proportions of the measured object 2020 so that ittakes up the same space as the physical object, and then place theobject on the situational awareness map of the SAC 302B. This processmay result in the 3D chair model, scaled and placed on the situationalawareness map, effectively filling in gaps 2050 in the S-MMSs 18Iawareness. In some embodiments, these objects may then be treated by anS-MMS 18I as if they were physically present in the world and properlyhandled by the situational awareness control 302B of an S-MMS 18I. Thisdata may be stored in local memory 120 and shared with the previouslydisclosed remote server or service used for public mapping 1510.

Socialization can be a challenge for MMS users, regardless of abilitiesor impairments. The act of mapping locations could be gamified toachieve both the complete mapping of locations more quickly and toencourage social interaction for S-MMS 18I users. Most locations havehigh- and low-trafficked areas, and, motivated by gameplay, thecompleteness of mapping could be encouraged and rewarded. In oneembodiment, a remote server 1510B configured to generate public map data1704 may enact rules 1708 in a rules engine 1706 which flag areas of lowmap 1802 quality. These areas may then be displayed to S-MMS 18 usersvia a paired smart device 1502, to users or caregivers via a webinterface 1542, and to others via smart devices 1532 in order toencourage mapping of those areas. In an embodiment, players may earnpoints/experience/status/achievements/rewards based on where they map,how much total area they map, if they are the first to map an area, ifthey identify a new obstacle or accessibility feature, etc. Additionallyor alternatively, the areas flagged by the remote server 1510B may becommunicated to S-MMSs 18I proximate to the area of the flag. Thisinformation may be received via one or more wireless transceiver 216onboard the S-MMS 18I, and communicated to the S-MMS controller 110B viaCNI 371. In some embodiments, this may cause the S-MMS 18I to alert theuser via the HMI 352 or a paired smart device 1502 that a mappingopportunity is nearby. Additionally or alternatively, the sensor tasker520 function of the SAC 302B may cause the S-MMS 18I to take additionalsensor readings when passing through a flagged area and communicate thatdata back to the remote server 1510.

A remote server 1502 may be configured to transmit messages/data andreceive messages/data to cause the games to be played on a receivingsmart device 1502, including for a user interface to be displayed on thesmart device to display data for the game and allow inputs to the smartdevice for the game (such as the inputs described above) that aretransmitted to the remote server for processing and response so the gamecan be played. Alternately, the games may be hosted in the hosted 125 orcomplex application space 130 of the S-MMS 18 (FIG. 1), and the S-MMScontroller 110B then would be configured to transmit messages/data andreceive messages/data to cause the games to be played on a receivingsmart device 1502, including for a user interface to be displayed on thesmart device to display data for the game and allow inputs to the smartdevice for the game (such as the inputs described above) that aretransmitted to the remote server for processing and response so the gamecan be played.

Augmented reality (AR) may enhance gameplay to offer 3D games, such asPacman, chasing aliens, or playing hide-and-seek with animals, inlocations that need to be mapped. This ability may encourage movementinto certain flagged areas for mapping, or may just enhance theenjoyment of the activity itself. FIG. 21 depicts an S-MMS 18M paired toan AR device 2102 via a wireless link 2104. In one embodiment, the CNIfunction 371 (FIG. 3) is provisioned such that at least one wirelesssystem is available to transmit and receive (transceive) data relevantto a desired operation of a mobile chair S-MMS 18M. In one embodiment,an AR device 2102, such as a smart device running an AR application or awearable such as MicroSoft HoloLens, communicates 2104 with the S-MMS18M via a communication processor 216 (FIG. 2) of the S-MMS 18M throughone of 802.11, Bluetooth, Bluetooth Low Energy (BLE), Near FieldCommunication (NFC), and/or Zigbee™, among others. These communications2104 are typically predicated on a “just works” model, one that requireslittle interface with the device 2102 and S-MMS 18M and the underlyingtechnology. The levels of security and privacy are device 2102, OS, andwireless service dependent. In some embodiments, the AR device 2102 maycommunicate 2104 with the S-MMS 18M via a wired interface. In someembodiments, the communications 2104 may be wireless and accomplishedvia a secure Bluetooth connection based on the HDP and MCAP standardsincluded in the Bluetooth Core Specification herein incorporated byreference.

Other activities may also, or alternatively, be gamified. For instance,completing regular exercises/physical therapy sessions or improvementsin such activities could be rewarded in a game or game-like fashion.These game applications may be hosted in the hosted 125 or complexapplication space 130 of the S-MMS 18 (FIG. 1) and communicate with theS-MMS controller 110B via an API 135. Activities that may normally betedious or monotonous may be gamified to encourage the users to performand to motivate them to excel with variouspoints/experience/status/achievements/rewards. In some embodiments, therewards may be one or more of based on the particular gaming environment(for the activity or chosen by the user), picked by the user from anassortment of different rewards, and personalized to motivate aparticular user. Feedback, prompts, and input for the games may beprovided via the HMI 352 or via a paired smart device 1502.

S-MMS user data may be characterized as HIPAA medical data or may becharacterized as non-medical data. By default, all user data associatedwith a user account may be characterized as private and secure,including gaming and mapping data. However, this data may be shared inmultiple ways. In some embodiments, a user may elect to share collectedmap data with a third party such as a mapping service or S-MMSmanufacturer. This data may be disassociated from the user when shared.In some embodiments, the user may choose to share only certain mappingdata on a transactional basis with third parties. For conditional ortransactional data sharing, technologies such as block chain may allowfor secure sharing and/or payment for data services provided. In someembodiments, the user and/or caregiver may set privacy settings andpreferences globally in the system and/or per use, per application orapplication type. In these embodiments, the S-MMS controller 110B and/orsmart device 1502, as the case may be, would implement theabove-referenced technology in the messages/signals/communications/datait sends.

Location Memory and Auto Return

MMS users often frequent the same locations inside and outside the home.The effort and time of navigation to these frequent locations increasesgreatly for mobile chair MMS users and can increase more for those withcognitive or mobility impairments that impact the motor functions neededfor steering. Simple navigation to frequent locations in one's own homecan become both time-consuming and taxing for some MMS users, whichdirectly impacts quality of life.

The best route system, previously discussed, can assist users withautonomous navigation to frequent locations within a home, facility, orother frequently visited place. In some embodiments, the S-MMS 18situational awareness controller 302B has an awareness of predeterminedlocations or a history of recently visited locations. The S-MMScontroller 110B uses the SAC 302B and navigation 363 to assist the userand speed their travel to a selected location. Users may select adestination using the S-MMS HMI 352 and/or the HMI of an associatedsmart device 1502. HMI selection mechanisms may include a control buttonor input on the HMI, a voice command, or the selection of a destinationfrom a list of predetermined locations. Additionally or alternatively,the S-MMS navigation 363 or drive path manager 529 of the S-MMScontroller 110B may make a rules based determination based on currentlocation, direction, and possible destinations of possible locations tooffer the user. Once a selection is made, the S-MMS 18 assists safenavigation to the selected destination, helping speed up the trip andlowering the burden of driving on the user. Autonomous navigation ismanaged by the drive path manager 529 of the SAC 302B based oninformation received from navigation 363 and other S-MMS systemspreviously disclosed. The motor controller 351 responds to commands fromthe SAC 302B to drive motors and actuators.

The S-MMS 18I may learn from the user's behavioral history and may beginto anticipate the user. In some embodiments, S-MMS 18I navigationhistory may be sent to a remote server 1510B where one or more of arules engine 1706 or machine learning engine 1710 may identify and flagfrequently visited locations. This information may be stored as privatedata 1702 on the remote server, or it may be transferred to S-MMS 18Ionboard memory 120. In an embodiment, the rules engine 1706 or machinelearning engine 1710 may be hosted on the S-MMS controller 110B. In anembodiment, the drive-path-manager may look for flagged locations in theprivate data 1702 proximate to the current path. Those locations maythen be offered to the user for selection via the HMI 352 or a pairedsmart device 1502. In some embodiments, the rules engine 1706 may lookfor correlations between locations and other factors, such as time ofday. For instance, if a user regularly travels to the kitchen for lunchat 12 PM, the S-MMS controller 110B may guide them there when the userturns in that direction around that time period. In some embodiments,the user may be presented with the suggestion by the S-MMS controller110B and choose yes or no to activate the automatic guidance or not.Based on the previously disclosed approach, the S-MMS controller 110Bmay be capable of determining when the user is differing from theirusual behavior, such as if the S-MMS controller detects the location asother than home at 12 PM, and the S-MMS controller will not suggesttravel to the kitchen at that time.

Motion Memory and Parking Assistant

Motorized mobile systems can be difficult to repeatably navigate due tothe imprecision of the steering mechanism (e.g. a joystick or steeringbar) and the lack of an ability (for some systems) to move directlysideways. In addition, frequent locations for parking often have arelationship to another object and/or activity, including parking nextto a toilet to allow toileting, parking in a van or truck in order to betethered to anchor points, and parking next to a bed, among others.These frequent parking locations can require precise position andmaneuvering to provide the appropriate proximity for access or the exactlocation for tethering in the vehicle scenario. Precision is requiredfor access for the desired location, but the proximity often requirescareful maneuvering. This precision movement and alignment is importantand time-consuming.

An S-MMS 18 may aid with parking at common locations by assisting withspeed and alignment. Assistance, in some embodiments, may include fullautonomous positioning and in other embodiments may simply constrainuser inputs to within a predefined error term of an ideal trackestablished by the drive path manager 529 of the SAC 302B. In anembodiment, an S-MMS controller 110B may utilize an SAC 302B (FIG. 5)along with one or more of the previously disclosed precision navigationtechniques, location memory and precision map data to navigate to apredefined parking location. This embodiment requires that the S-MMScontroller 110B stores the required parking orientation at thatlocation. This data may be stored either onboard S-MMS 18 memory 120 oron a remote server 1510. By combining the previously disclosedtechniques, a parking assistant implemented by the S-MMS controller 110Bmay eliminate the need for the user to precisely maneuver by automatingtasks. The S-MMS 18 prevents misalignment through its understanding ofthe correct positioning at that destination, and speeds parking byrecognizing the safe path to, and any obstacles near, the desiredparking location. In some cases, the S S-MMS controller 110B may enteran automated parking mode where the S-MMS controller takes over andcontrols the S-MMS 18 to park more quickly and precisely and reducestrain on the user. The automated parking mode may store informationabout common parking areas and/or it may be able to perform commonactions in known and/or new locations using sensor data.

FIG. 22 illustrates a typical path and the motions required tosuccessfully park a mobile chair S-MMS 18N inside an accessible van2220. In an embodiment, a parking assistant implemented by the S-MMScontroller 110B may respond to external, wireless signals and maytransmit one or more wireless signals via one or more communicationsprocessors or transceivers 216. Using the parking assistant, a user maytake advantage of many of the capabilities of the disclosed S-MMS 18 ina fluid process. As the user approaches the van 2220 they may beprompted via the HMI 352 or a paired smart device 1502 to confirm thatthey are heading to the van 2220. The SAC 302B of the S-MMS controller110B may recognize the van 2220 based on user input, image recognition,location, patterns of use, and/or wireless ad hoc communication (e.g.such as WAVE™ as previously disclosed). If parking mode is confirmed,the S-MMS controller 110B may transmit one or more wirelesscommunications 2230 to a control system of the van 2220 to cause one ormore of automatically unlock the van if the user is authorized, open thedoor, and lower the automatic ramp 2240. Wireless communication mayinclude an ad hoc network or RF signals. As the ramp 2240 touches down,the S-MMS 18N user may confirm that they wish to use the parkingassistant and the S-MMS controller 110B may cause the S-MMS 18N toautomatically drive up the ramp 2240 and follow the path and motions2250 depicted to position itself relative to attachment point(s) 2260 inthe vehicle. The path and motions may be calculated and executeduniquely each time by the drive path manager 529 of the SAC 302B toposition the S-MMS 18N relative to one or more objects or referencesavailable at the location. As a non-limiting example, one or more of theimage sensors may be tasked by the sensor tasker to monitor for theattachment point(s) 2260 (e.g. tie down straps) using feature extraction530. These “landmarks” may then be used to develop the drive path 2250to be used by the -MMS controller 110B. Additionally or alternatively,the SAC 302B may be configured so that once the S-MM 18N is positionedin a location proximate to the target, the S-MMS controller 110B, incoordination with the motor controller 351, executes a predefined seriesof motions retrieved from memory 120.

FIG. 23 illustrates the use of one or more optical or retro-reflectivetargets 2330 to assist in S-MMS 18N positioning. Placing targets 2330allows the S-MMS 18N to position itself quickly and with high precisionin a wide variety of settings. As an added benefit, targets 2330 may beused as an approximate reference point for positioning for non-MMSusers. As a non-limiting example, a vinyl sticker 2330 with a predefinedpattern may be placed at particular coordinates relative to a handicapaccessible toilet 2320. Data may be associated with the particularpattern or pattern and location combination. The unique pattern may beassociated with a location, a function, a message, or set of desiredS-MMS 18N actions that may assist in positioning the S-MMS 18N inrelation to the target 2330. This information may be stored onboardS-MMS memory 120, may be uniquely associated to an individual user andstored as private data 1702, or maybe accessible public data 1704retrieved by the SAC 302B from a remote server 1510.

The -MMS controller 110B is capable of recognizing the positioningtarget 2330. In one embodiment, one or more onboard image sensor reports374, received by the SAC 302B, are processed in feature extraction 530,and the target pattern is recognized by one or more of the patternrecognition processes previously disclosed (FIG. 19). Positioning maystill be assisted by use of other onboard sensor reports 372-376 (FIG.4) as well as data referenced or provided by the target 2330. In anembodiment, SAC 302B sensor fusion 500 associates the target 2330identified by feature extraction 530 as part of the object track, inthis case a toilet 2320. This association of the target and the objectdata serves as an alternative approach to dual mode variance, wherein apositioning target proximate to the object is used as an extension ofthe object to provide information on the location or a track to the SAC302B and increase the accuracy of the situational awareness mapmaintained by the tactical manager 527.

Targets 2330 may take many forms and include standardized targets thatare used in public settings to increase accessibility for S-MMS 18Nusers. Additionally or alternatively, custom targets 2330 may be usedthat users may purchase or print and place at locations of theirchoosing to assist with positioning tasks of their choosing. In someembodiments, the targets 2330 may be recognized objects in thesurroundings, such as a particular piece of furniture or decoration in ahome rather than a sticker. In some embodiments, targets 2330 may takeany form that is recognizable by the S-MMS controller 110B of the S-MMS18N, including certain objects, artwork, stickers, decals, and patternssuch that the targets may blend with the overall aesthetic of thelocation they are in.

In some embodiments, targets 2330 may incorporate or be replaced by RFIDchips, Bluetooth beacons, or other computer readable chips, such thatthey may be recognized by, and transmit their coordinates to, the S-MMScontroller 110B wirelessly as previously disclosed for general,precision indoor navigation.

Transfer Assistance

Transfers to and from an MMS are common and critical for users. As anon-limiting example, therapists spend months with new mobile chair MMSusers breaking down transfers, moving into a car, moving to a toilet,and/or switching to a couch or chair. Each piece of that journey becomespart of a routine that, when improperly done, can lead to seriousinjury. If an MMS user's seat is positioned slightly wrong for atransfer, it changes distances and balance points that they havepracticed and rely on. These changes often lead to falls. But even ifthe user doesn't fall, they may strain or pull muscles in the attempt tocorrect for an initial positioning error.

A transfer mode allows S-MMS 18N users to have more control andconfidence when transferring to and from their S-MMS. FIG. 24illustrates a common transfer situation. As a non-limiting example,consider a user positioning their S-MMS 18N next to a chair 2420 thatthey plan to transfer to. This situation is unique from tables anddesks, because rather than pulling forward, toward the interface, usersoften approach from the side and then transfer sideways to the chair2420. The user must position the S-MMS 18N seat 2440 both vertically (h)and horizontally (d) in relation to the target chair 2420 with highaccuracy. The parking assistant, as previously disclosed, may assist theS-MMS 18N user in accurately achieving the desired horizontal distance(d). Using the transfer mode, the S-MMS controller 110B mayautomatically adjust the seat position (e.g. by sending control signalsto the motor controller 351 after determining the height from sensorreports 371-374) to match the vertical height of the adjacent seatingsurface 2430 or maintain a predefined vertical height difference (h) perthe user's settings for easier transfer. Seat height of the adjacentseating surface may be determined using one or more onboard, non-contactsensors 372.

In an embodiment, as the S-MMS 18N approaches the chair 2420, or othertarget transfer surface, the user may select transfer mode via the S-MMSHMI 352 or a paired smart device 1502 and indicate the transfer target2420. With this information, the S-MMS controller 110B SAC 320B may sendcontrol signals/commands to the motor controller 351 causing the S-MMS18N to autonomously pull within a maximum preferred transfer gap (d) ofthe transfer target 2420. This gap (d) may be unique to each user andstored in memory 120 as part of a unique user profile. In addition, theS-MMS controller 110B may position the S-MMS 18N per settings inrelation to the transfer target 2420 front to back. The S-MMS controller110B may then adjust the seat 2440 height in relation to the height ofthe adjacent seating surface 2430 for easier transfer. In one example,the S-MMS controller 110B repeatedly receives and processes sensorreports to determine a current distance away from the transfer target2420 and transmit control signals/commands to the motor controller 351causing the S-MMS 18N to autonomously pull within the maximum preferredtransfer gap (d) of the transfer target 2420. Similarly, the S-MMScontroller 110B repeatedly receives and processes sensor reports todetermine a current height of the seating surface 2430 of the transfertarget 2420 and transmit control signals/commands to the motorcontroller 351 causing the seat 2440 height to be moved to the correctheight relative to the adjacent seating surface 2430. In one embodiment,additional non-contact sensors 2450, such as an infrared or ultrasonicsensor, is mounted in a location such as the armrest of the chair sothat the height of a transfer seating surface 2430 may be measuredrelative to the S-MMS 18N seat 2440. Most S-MMS users prefer to transferfrom a higher to a lower seating surface, so the user may preprogram apreferred height differential (h) so that the S-MMS controller 110B ofthe S-MMS 18N provides a personalized optimal transfer when the mode isactivated Similar to the previously disclosed approach mode, the S-MMScontroller 110B of the S-MMS 18E may also be programmed and the settingsfor common transfer locations can be saved in local 120 or remote 1510memory, in some embodiments.

Additionally or alternatively, the transfer mode may be configured toautomatically modify other seat settings and/or deploy transfer aides.In order to ease the transfer process, transfer mode on the S-MMScontroller 110B may be configured to automatically control any poweredsystems available on the S-MMS or via the motor controller 351. Forexample, for S-MMSs 18 that have powered, self-lifting arm rests,activation of the transfer mode may automatically raise the armrest thatis directly adjacent to the selected transfer target. S-MMS 18 withactive suspension systems may be configured to have the S-MMS controller110B automatically lower or stiffen the suspension of the S-MMS 18 toassist in stability during the transfer. In some embodiments, S-MMSs 18may include transfer assistance devices, such as extra grab bars orsliding support rails. When available, use of such features may belinked to the transfer mode being engaged, and their activation may becontrolled by control signals sent from the S-MMS controller 110B.

Once a transfer is complete, the transfer mode may automaticallyreconfigure the seat to settings selected for transfer back into theS-MMS 18N. The S-MMS controller 110B may confirm that the transfer iscomplete by ensuring there is no user present. This may be accomplishedby confirming that no weight or force is sensed on the S-MMS 18N basedon user sensor reports 375 received by the SAC 302B, by trackingproximity of a paired wearable sensor in communication with the CNI 371,or any other means. A user may set the S-MMS 18N via the S-MMScontroller 110B to lower the height of the seat to just below the heightof the surface they are transferring from for ease of transferring backto the S-MMS 18N at a later time.

Approaching Objects and Furniture

For many MMS users, collision is a way of life. Tables, desks, chairs,and most of the objects in their daily life have been designed withoutthem in mind. Even good drivers run into things, often damaging theirMMS and their surroundings. What isn't often considered is the toll thatthis takes on users physically as they smash toes, knees, and fingers onthe objects they run into.

It is not always enough for an S-MMS 18 safety system to behave inaccordance with the spatial requirements of the S-MMS 18 itself andavoid hard collisions. In some embodiments, the S-MMS 18 may also takethe user into consideration. As an example, positioning a mobile chairS-MMS 18 in front of a table requires the user to pull up to the rightlocation (front-back) with high precision. This is often difficult withexisting steering HMIs 352 like joysticks. In addition, since tableheights differ, the user may have to position the S-MMS 18 seatvertically (up-down) to the right level as they approach the table.Often switching between positional control (front-back, left-right) andseat height control (up-down) requires multiple user button presses andactions via the HMI 352. If any of these directions are inaccurate, thenthe user's hands, fingers, or knees may run into the table. Even worse,often times they will get pinched between the table and the S-MMS 18.This is a simple example of existing conditions for many MMS users, andit ignores other user preference settings like seat back positioningwhich may be different for setting at a desk versus at a dinner table.

An approach mode on an S-MMS 18 allows the user to get close enough touse or interact with common objects (e.g. desks, tables) as needed, butprevents injury of the user or damage to the object by preventingcontact on both approach and exit of the S-MMS. This feature isillustrated in FIG. 25. As the mobile chair S-MMS 18N approaches atable, desk, bar, or other object with an overhang 2520, the situationalawareness controller 302B of the S-MMS controller 110B detects theobject 2520 based on one or more sensor reports 371-374 and positionsthe seat height to match the object height (generally per user history,general recommendations, and/or user preferences associated with a userprofile stored in memory 120). This is unique from a typical collisionavoidance system that would simply stop the S-MMS 18N user fromapproaching the table or other object 2520. For this reason, the usermay need to activate the approach mode behavior either as an always onfeature or per event via a voice command, gesture, or other HMI 352input. This feature frees the user to focus on positioning the mobilechair S-MMS 18N close enough to the table or desk 2520 without worryingabout injury.

For convenience, and because users will tend to use the same tables anddesks or other objects 2520 often, the positioning system can be set toremember specific locations or types of objects as previously disclosed.As the S-MMS 18N approaches a known location or object type, the S-MMScontroller 110B may auto adjust based on memory 120 without userintervention.

The S-MMS controller 110B can remember both preferred seat settings andpreferred front-back position to the object 2520 so that the S-MMScontroller may automatically stop at the preferred spacing and adjustfor comfort of the user. Approach mode may rely on a combination ofsensors previously disclosed. Additionally or alternatively, additionalsensors 2502 or 2504 may be utilized. In some embodiments, one or moresensor reports (e.g. 2502) may be provided to the S-MMS controller 110Bby one or more sensors embedded in a pair smart device 1502.

In some embodiments, the S-MMS controller 110B may take intoconsideration both the extents of the S-MMS 18N and the location of thedriver's knees, fingers, elbows, and other leading features (e.g. viadata of those locations stored in memory and retrieved by the S-MMScontroller, by sensors on the S-MMS placed at or focused on thoselocations as described herein and transmitting sensor reports of thoselocations to be used as inputs in the processes of the S-MMS controller,or otherwise). If a problem is detected by the user at any point, theycan take over control of the S-MMS 18N. In some embodiments, a verbalcommand such as “stop” may halt the automated activity at any point inthe process. To further safeguard the user, additional safety checks maybe activated during approach maneuvers that are intended to sense andavoid soft impacts. Sensors on the seating system, or other locations,may sense if a force is detected in a place where force is not expectedduring a maneuver and pause the maneuver.

FIGS. 26A and 26B illustrate a non-limiting example placement of suchsensors 2630 and 2660 to measure forces on the arm rest 2610 and footrest 2640 of a mobile chair S-MMS 18N. In the event that a soft impactforce F is detected, the S-MMS 18N may halt or slowly back away from theobject being approached, which may impact one or more of those areas,and await further instructions from the user. Users may set preferencesfor how the S-MMS 18N should react in such situations. Thisfunctionality can also be linked to user health systems, such aselectrodermal activity tracking devices or heart rate devices capable ofsensing user pain if available and paired with the S-MMS 18N.

The approach mode may be further enhanced on S-MMSs 18 equipped withmotorized accessories. As a non-limiting example, FIGS. 27A and 27Bdepict a self-retracting control unit 2710. The depicted exampleself-retracting unit 2710 moves out of the way automatically when theapproach mode is engaged. In this example, a sensor on theself-retracting unit 2710 or a device attached to the self-retractingunit transmits one or more sensor reports to the S-MMS controller 110B.The S-MMS controller receives and processes the sensor reports from theself-retracting control unit 2710 to determine the self-retractingcontrol unit will impact an object 2720 and transmits a controlsignal/command to the self-retracting control unit instructing theself-retracting control unit to retract. The self-retracting unit 2710receives and processes the control signal/command from the S-MMScontroller 110B and, in response thereto, retracts an arm or othermember of the self-retracting unit. This keeps the user from having tomanually reposition the unit in order to pull in close to a table ordesk 2720. It also provides a clear indicator that the autonomousapproach mode has been engaged. As a further benefit, the actuators usedto move the unit may also be used to sense any unexpected forces thatmight indicate pinching or soft impacts enhancing the previouslydisclosed safety checks for approach mode.

Call and Send to Charge

An S-MMS 18 may be its user's access point for their entire world, bothinside and outside the home. As a non-limiting example, a user of amobile chair S-MMS requires their S-MMS to be within a certain proximityto transfer off and onto the S-MMS. Whenever the user is not on theirS-MMS, it must be close by when the user wants to change locations, orthe S-MMS must be brought to them by someone else. Not being inproximity to their S-MMS puts the user at risk if mobility is needed forsafety reasons and degrades the quality of life and independence of theuser if they cannot attend to their personal needs or desires becausethe location of the S-MMS does not allow transfer.

Due to the importance and frequency of transfers, a mobile chair S-MMS18N may be programmed with the needed proximity to the user for asuccessful transfer (e.g. using the previously disclosed transfer modefunctionality), which would allow the S-MMS to safely gain thisproximity to a previously visited or preprogrammed location based on thepreviously disclosed auto-return functionality combined with precisionnavigation capabilities of the S-MMS controller 110B. Combining thesefunctions, a user currently not in their S-MMS 18 may command theirS-MMS to come to their location and park within the proximity requiredfor transfer. These commands may come in many different ways, includingvoice, direct input, hand signal, or other methods of input through theHMI 352 or a paired smart device 1502.

The location of the user may be determined by the S-MMS 18 in a numberof manners. The simplest manner being data stored in a memory 120 of thelast time that the user was detected on the S-MMS 18 by the SAC 302B,e.g. based on user sensor reports 375. As a non-limiting example, weightsensors embedded in the seating system may add a waypoint to the S-MMSsituational awareness map at the location that user weight is no longerdetected in the seat. Additionally or alternatively, a location of theuser may be determined based on the location of a wearable device orother device configured to transmit a beacon signal for detection by oneor more communication processors 216 on the S-MMS 18. In an embodiment,the S-MMS controller 110 may use signal strength of the beacon to detectthe user.

Managing the charge of battery powered S-MMSs 18 is criticallyimportant. The safety of the user relies on a fully functional S-MMS 18,which must be sufficiently charged to function as designed. FIG. 28Adepicts a charging station 2802 located across a room from a bed 2804.In this example, the location of a charging station 2802 may not bewithin a safe proximity for the user to transfer to another location(e.g. the bed 2804) while the S-MMS 18N charges. In one embodiment, thelocation of the charge station 2802 has been marked by the user on theirprivate map data 1702 stored on a remote server 1510 or onboard 120.Based on one or more of the navigation and location determinationtechniques previously disclosed, the S-MMS controller 110B navigation363 function is aware of the S-MMS 18N current location and itsproximity to the charging station 2802 and may control the S-MMS totravel to the charging station from the bed 2402 and/or to the bed fromthe charging station at a preset speed or an optional speed determinedby input to the HMI 352 (e.g. by causing one or more commands to be sentfrom the S-MMS controller to the motor controller 351).

As illustrated in FIGS. 28A and 28B, the user may transfer to their bed2804 at the end of the day and then command the S-MMS 18N to go to itscharging station 2802 from its current location. In the morning, theuser may command the S-MMS 18N to return to the bed 2804 within aproximity required to transfer back onto the S-MMS 18N. When paired witha wireless or robotic charging system, this functionality allowssignificant freedom and confidence for the user. These commands may beinput using any one or more different methods, including voice, handsignal, or other methods of input through the HMI 352, paired smartdevice, wearable, and/or application 1502.

Avoiding People and Animals

The disclosed systems and methods allow an S-MMS 18 to avoid people andanimals. Consider that motorized mobile systems often navigate and movein a distinct way, at a different speed, and occupy a very differentfootprint from the people or animals around them. This unique behavior,combined with the power and weight of a typical MMS, make awareness andavoidance of people, animals, and other things that move on their ownimportant to the safety of both the S-MMS 18O user and those movingaround it. As opposed to simply avoiding collisions with otherautomobiles in generally open environments like an automobile, the S-MMS18O needs to watch for and avoid toes and tails, sometimes in tightlyconstrained spaces. S-MMS collision avoidance systems may be much moreaccurate than existing automobile systems. The disclosed SAC 302B (FIG.5) combined with the dual mode variance techniques disclosed (FIG. 12)and careful attention to frames of reference and conversion error (FIG.13) lay the foundation for collision avoidance with the requiredaccuracy for an S-MMS 18.

S-MMSs 18 are often used in environments where people are in extremelyclose proximity. People can walk erratically and, when moving around anS-MMS, can be positioned where the user cannot see them or their toes.Depending on the abilities of the user, they may not have an awarenessof those around them as they navigate their S-MMS 18 because of manyfactors, like a person who is moving quickly to pass them, is sitting,is short, is a child, or has moved from one side of the S-MMS toanother. The disclosed SAC 302B with tactical manager 527 and threatassessor processes 528 (FIG. 5) enhance situational awareness of theuser by creating and maintaining a situational awareness map that canthen be used by the drive path manager 529 and alert manager 540.

In some embodiments, the S-MMS controller 110B is aware of people andanimals (unique tracks) positioned or moving close by. The SAC 302B onthe S-MMS controller 110B may recognize movement, and cause the motorcontroller 351 to react to that movement in the safest way possible,including slowing, stopping, or changing direction of the S-MMS 18.Additionally or alternatively, the user may be alerted via the HMI 352of the location of people, animals, and objects positioned close by. Asa non-limiting example, the SAC 302B on the S-MMS controller 110B mayslow down the S-MMS 18 to protect individuals in the vicinity.Additionally or alternatively, the alert manager 540 of the SAC 302B mayemit audible or visual warnings, using HMI 352 provisions such as lightsand speakers, to those around the S-MMS 18 to make them aware that thedevice is within close proximity.

Wireless Identification

FIG. 29 illustrates an embodiment of an S-MMS 18O in which a non-user(e.g. caregiver, service animal, technician, or other partner) may weara wearable device 2904, such as a wrist band, necklace, ring or otheraccessory, with an embedded transceiver configured to communicatewirelessly 2902 with the S-MMS controller 110B. Additionally oralternatively, a caregiver or other individual may use their smartdevice (e.g. watch or smart phone) as the “accessory” if equipped withthe proper transceiver and application. The wearable device 2904transceiver may be configured to use one or more of the following meansof communication 2902:

-   -   Active or passive RF,    -   Bluetooth,    -   IEEE 802.11 based Wi-Fi communication,    -   multiple low-rate wireless personal area networks (LR-WPANS)        based on IEEE 802.15.4 including ZigBee, MiWi, and Wireless        HART, or,    -   near field communications (NFC) protocol based on ISO/IEC 18092.        The applicable standards are herein incorporated by reference in        their entirety.

As a non-limiting example, a caregiver, may wear a wrist band wearable2904 with an embedded RF transceiver. The transceiver may be passive oractive, with active transceivers allowing longer-range communication butrequiring a power source, such as a battery. When the wrist bandwearable 2904 comes within range of the S-MMS 18O, the signals 2902 fromthe wrist band are received by a communications processor 216 onboardthe S-MMS 18O and the contents of the signal may be routed to the S-MMScontroller 110B via a security processor or arbitrator 212. The detectedpresence of the wrist band wearable 2904 may cause the SAC 302B tooperate uniquely.

Animal Partner Example

An S-MMS 18O may respond uniquely to individual people or animals and ata level of refinement not typically needed by existing autonomoussystems. This may be accomplished by marking their tracks on thesituational awareness map maintained by the SAC 302B and associatingtheir track with unique threat assessor 528, drive path manager 529, orcollision manager 526 behaviors on the SAC 302B (FIG. 5). As anon-limiting example, service animals are common companions to S-MMS 18users. Socialization, interaction, and finding community can bechallenging for those with limited mobility, and service animals manytimes provide companionship to those who use an S-MMSs. There arespecial training programs to prepare service animals to safely workaround mobile chair MMSs. However, an S-MMS 18O may also be trained towork as a team with a service animal. In some embodiments, the S-MMS 18Orecognizes its service animal partner and treats them differently thanother animals or people. One example of this behavior may include theability to for the S-MMS 18O to follow the service animal on command.

An example, illustrated in FIG. 30, is for the service animal 3010 tohave the ability to “nudge” the mobile chair S-MMS 18O and redirect itby taking specific positions (say getting close to one side of the S-MMS18O) while the user is driving. This behavior is unique and may only beengaged when a known service animal is recognized. Service animalcontrol mode may be turned on or off by the user via the HMI 352, insome embodiments. In some embodiments, it is engaged automatically whenthe animal 3010 is in range. The S-MMS 18O may recognize the serviceanimal by a special smart collar or addition to the service animals'vest 3030. The collar or vest 3030 may include a transponder ortransceiver which uses one or more of the wireless protocols previouslydisclosed to transmit a signal to the S-MMS 18O. S-The MMS controller110B receives and processes the signal to determine one or more actionsare to be taken (e.g. move the S-MMS forward, backward, left, or right,or generate an alert for example) and transmits one or more controlsignals/commands to the motor controller 351, HMI 352, and/or otherS-MMS 18O component as described herein to cause the one or more actionsto be taken. Additionally or alternatively, the S-MMS controller 110Bmay use onboard cameras to recognize people and animals visually usingon or more of the previously disclosed feature recognition approachesincluding use of badges or tags 2330 worn by the animal or individual3010. In one embodiment, an S-MMS 18O receives a signal from an activeRF transmitter embedded in a dog vest 3030 worn by the service animal3010 via one or more communication processors 216 on the S-MMS 18O. Thepresence of the service animal signal is communicated to the SAC 302Bvia CNI 371 which causes the SAC 302B to task image sensors 520 on theS-MMS 18O to take 360-degree data. Feature extraction 530 is used torecognize a unique pattern (e.g. a QR code) on the animal's vest whichhas been pre-associated with that animal. When recognized, sensor trackfusion 500 may associate the bearing location of the animal 3010 withrange data received from other sensors (e.g. 371-373) and associate thetrack with the animal's unique identity code or flag. The tacticalmanager 527 may then place the identified track on the situationalawareness map and the unique identity code may notify other functions(e.g. collision manager 536, drive path manager 529, threat assessor528) to react to the individual track of the service animal 3010 using aunique, preconfigured, set of control instructions or rules. This sameconcept of unique behavior for unique individuals may extend to the waythe S-MMS controller 110B reacts to specific caregivers, coworkers,trainers service animals, or other individuals. Unique behaviors mayinclude, but are not limited to, allowing individuals to get closer tothe S-MMS 18O before taking evasive action, following the individual,allowing the individual to nudge the direction of travel of the S-MMS18O, or modification of any behavior of the motor controller 351 or HMI352.

Turn and Look

For MMS users, social norms like maintaining eye contact and facing thespeaker during conversations may not currently be possible withoutassistance or manually maneuvering their MMS. This not only distractsfrom engagement in the conversation, but may be a challenge for userswho have diminished motor function or cognitive impairments that delayreactions. As illustrated in FIGS. 31A and 31B, the S-MMS controller110B of an S-MMS 18O may assist user engagement when conversing andinteracting by automatically orienting the user to face individuals withwhom the user is engaged. The drive path manager 529 of the SAC 302Ballows the S-MMS 18O to safely orient itself when in close proximity topeople or objects. Moreover, in some embodiments (FIG. 5) the SAC 302Bmay have an accurate situational awareness map of those in the immediatevicinity, their location, and their relative position to where the S-MMS18O (i.e. user) is facing. Turn and look functionality allows the S-MMS18O to be directed to face different individuals with minimal or noeffort on the part of the user.

When turn and look is enabled on the S-MMS controller 110B, the SAC 302Bmay use onboard cameras (e.g. image sensor reports 374) and microphones(e.g. non-contact sensor reports 373) to add information to the tracksof individual people, determining whether they are speaking, if they sayany key words, and to locate their faces. In an embodiment, microphonesassociated with one or more onboard sonic sensors may be temporarilyretasked to sample ambient noise. Additionally or alternatively, one ormore dedicated microphone sensors may be added to the S-MMS 18O for usewith this system. For example, the MMS controller 110B may receive andprocess a sensor report from a microphone to determine one or moreactions are to be taken (e.g. move the S-MMS in a direction to face thefront of the S-MMS toward the detected sound, for example) and transmitone or more control signals/commands to the motor controller 351 asdescribed herein to cause the one or more actions to be taken. Whencombined with proximity and location information already identified witheach individual's track on the situational awareness map maintained bythe tactical manager 527, this allows enhancement of the existingsituational awareness map of nearby individuals with information ontheir direction of interaction (which way they are facing).

In some embodiments, in its most basic form, the user may simply flicktheir HMI 352 input device (for example, a joystick) in the direction ofan individual they wish to speak with and the S-MMS controller 110B willreceive and process a signal from the HMI indicating the direction andreorient the S-MMS 18O to face the nearest individual in the chosendirection. HMI 352 input devices may include a series of buttons or ajoystick that is used to select individuals to direct the user towards.Another type of user interface is a map of the recognized individuals tobe displayed on the screen of a paired smart device 1502 so that theuser can simply select the person they would like to speak with.

Turn to look mode may alternatively or additionally have an automaticsetting, which with minimal or no input from the user automaticallyfaces the S-MMS 18O towards an individual based on one or morepredetermined criteria stored in memory 120. The S-MMS 18O may allow theuse of onboard microphones to detect a person's name and automaticallyturn the user to face that person, in some embodiments. For example, theMMS controller 110B may receive and process a sensor report from amicrophone to determine one or more actions are to be taken (e.g. movethe S-MMS in a direction toward the detected name, for example) andtransmit one or more control signals/commands to the motor controller351 as described herein to cause the one or more actions to be taken.This behavior may be filtered by voice code so that, for example, theS-MMS controller 110B will react when a mother or father calls the nameof a child in an S-MMS 18O but will not react similarly for anyone else.

Additionally or alternatively, the turn of turn and look functionalitymay be triggered by other signal sources such as light, sound, orwireless signals received from a certain direction. For any of thesesignal sources the signal may be recognized based on one or more ofpresence of the signal, signal frequency, sequence or pattern, orintensity.

Crowd Navigation

Navigating in a crowd is often particularly difficult in an MMS. Drivingan MMS effectively lowers a user's field of view, making it difficult tosee a destination past the crowd. This works both ways, so that thecrowd perceives the MMS location as a hole in the crowd. Adding to thechallenge of visibility as the crowd presses in, it can become extremelydifficult to maneuver the MMS without colliding with or injuring nearbyindividuals. For MMS users, crowds can be both dangerous andintimidating. What is needed are novel methods to allow MMS users toconfidently and safely visit crowded environments like sporting events,community events, and amusement parks.

The situational awareness controller 302B of the S-MMS controller 110Bmay assist in crowd navigation. The increased confidence in situationalawareness provided by dual mode variance accuracy (FIG. 12) and theother navigation approaches disclosed herein are critical to the abilityto operate and avoid collisions (e.g. collision manager 526) in atightly constrained crowd situation. By providing real-time groundmapping and dynamic anti-tip features, the stability manager 525 allowsS-MMS 18 users to focus on avoiding and following the crowd rather thanworrying about safety. This somewhat reduces the cognitive load on theuser. Furthermore, the stability manager 525 of the SAC 302B may allowusers of S-MMSs 18 with significant seat lift or stand capability todrive confidently within bounds defined by the drive path manager 529and enforced by the S-MMS controller 110B while elevated.

These things significantly aid the user, but do not fully solve theproblem. At a certain crowd density, the S-MMS tactical manager 527 andor threat assessor 528 processes may become overwhelmed by the number ofobjects, or tracks, in the environment or the number of unique objectsidentified may be so close together as to effectively form one object.Additionally or alternatively, even in less dense crowds, an S-MMS usermay want to simply follow along with the general flow of the crowd. Forthose reasons, a crowd following mode may be included in the S-MMScontroller 110B. While this and other modes or functions disclosed areembodied as included on the S-MMS controller 110B, it is clear thatthese modes and functions may alternatively or additionally be deployedvia an API 135 which communicates with the S-MMS controller and ishosted in one or more application spaces 125, 130.

FIGS. 32A and 32B illustrate an example of crowd interaction with anS-MMS 18P. Crowd following mode, when enabled, may cause the tacticalmanager 527 of the SAC 302B to group multiple detected threats, ortracks, into zone reports with associated kinematic properties on thesituational awareness map and create a map of “free space” 3202 aroundthe S-MMS 18P. This map of free space 3202 may be incorporated as partof the situational awareness map or may be maintained as a separateentity. Crowd following mode may then cause the drive path manager 529to engage an alternative set of flock based rules stored in memory 120.These rules may include one or more published flocking algorithms.Rather than trying to navigate to a specific location, the S-MMScontroller 110B may use onboard sensors to follow the flow of people andkeep the S-MMS 18P moving.

In one embodiment of flock based rules, the SAC 302B may establish andhold some basic rules. These may include matching the speed of the zonemovement in front of the S-MMS 18P, maximize/equalize space between theS-MMS 18P and the free space on either side, or other published flockingalgorithms. This mode may be triggered manually via the HMI 352 or apaired smart device 1502 or offered to the user when the tacticalmanager 527 or threat assessor 528 has determined that a significantnumber of tracks are being detected. An operator may choose to manuallyoverride the autonomous operations by the S-MMS controller 110B at anytime, in some embodiments.

FIG. 33 illustrates crowd navigation enhanced by integration of thepreviously disclosed best route system. Users may select both crowdfollowing and location based navigation systems at the same time (e.g.via the HMI 352 or a paired smart device 1502) so that the S-MMS 18P maymove towards a selected target location, along an accessible route,while navigating the flow of traffic in a crowd. As a non-limitingexample, an S-MMS user may select a target end location for the S-MMS18P to navigate to autonomously from a map on their paired smart device1502 while the S-MMS 18P is surrounded by a moving crowd of people. Thisselection is communicated to the S-MMS controller 110B at which pointthe SAC 302B drive path manager 529, in coordination with navigation363, determines the optimal accessible route (as illustrated in FIG.33). Additionally, the tactical manager 527 recognizes that the user issurrounded by a moving crowd and alerts the drive path manager 529 andthreat assessor 528 to switch to crowd navigation mode. Based on this,the drive path manager 529 may adjust its navigation approach. Ratherthan taking the ideal accessible route or following the direction oftravel of the crowd (Traffic D_(TRAVEL) in FIG. 33), the drive pathmanager 529 may cause motion across the main flow of traffic in a crowdby adjusting standard flocking rules. In an embodiment, the drive pathmanager 529 may cause one or more signals/commands to be transmitted tothe motor controller 351 to cause the motor controller to engage thesteering and drive of the S-MMS to drive on the edge of the detectedfree space 3202A in the direction of desired travel in order to createspace in the crowd. In some embodiments, this edge following behaviormay also be activated when the user inputs manual steering inputs intothe HMI 352 while in crowd mode.

As a way to enhance the user experience, an amusement park or stadiummay integrate their navigation, mapping, and crowd management systemswith the S-MMS 18P through an API. This both enhances the userexperience via a more accurate and easy way of finding information fordestinations and enhances the facilities management by using S-MMSs 18Pto provide real time crowd density and flow information.

Auto Queueing

Queuing can be another tedious task for MMS users. Parks, stadiums, andother facilities can simplify this process by providing auto-queuingguidance to S-MMSs 18, for example, through an application loaded inmemory 120 and accessing the S-MMS controller 110B via an API 135. Insome embodiments, the park may install special symbols such as a machinereadable QR code, floor or wall markings, or a coded sequence of lightsthat instructs S-MMSs 18 as to how they should behave in a queue. In oneembodiment, an S-MMS 18 equipped with one or more cameras may usefeature extraction 530 on the SAC 302B to evaluate image sensor reports374 for special symbols. When recognized by feature extraction 530,sensor track fusion 500 may alert the drive path manager 529 and causethe S-MMS controller 110B to send one or more control messages to themotor controller 351 and/or HMI 352 to cause the motor controller toengage the steering and drive of the S-MMS and/or cause the HMI toprovide feedback or take another action. Additionally or alternatively,auto-queuing information may be transmitted wirelessly to the S-MMSusing one or more of the previously disclosed methods, such asBluetooth. This allows a user to cease manual operation while queuingand have the S-MMS controller 110B simply and automatically guide theS-MMS 18 through the queue (e.g. onto a ride or into a stadium). In someembodiments, S-MMSs 18 may be guided through queues by other wirelessmeans, such as those disclosed for wireless ID of individuals andanimals and or turn and look functions.

For locations where auto-queuing as previously disclosed might not beavailable, the user may pull into a queue and then select a feature tofollow a queue autonomously. FIG. 34 illustrates an embodiment of thisfunctionality. In an example, many queues are separated by fabric lines3420. In an embodiment, an S-MMS 18P user may select queuing mode viathe HMI 352 or on a paired smart device 1502 when entering a queue. Viaone or more of the input methods (for example using a touch screen on apaired smart device) the user may select the fabric line 3420 from animage. Based on their selection, the drive path manager 529 of the SAC302B may combine a wall-following algorithm based on the selectedreference with collision avoidance functionality (e.g. collision manager526) which maintains a distance to d₁ to the next person in the queue3410 and a distance d₂ to the divider 3420 to provide hands-freeoperation in a queue. The selected reference may be recognized byfeature extraction 530 from image sensor reports 374 using one or moreof published edge detection, feature recognition, and/or colorrecognition techniques.

In some embodiments, S-MMS users may be guided through a facility,queue, ride or other activity by augmented reality overlays (FIG. 21)provided by a facility so that navigation may still be partially orcompletely manually controlled by the user within one or moreparameters.

Augmented, Situationally Aware Reality

The mobility MMSs enable freedom for their users. This freedom ofchoice, and the independence it enables, comes with a requirement of newskills and responsibilities. As an example, new users of an MMS now havecontrol of a device that can cause injury to others, and they mustdevelop new skills to operate the MMS safely. Users who struggle todevelop the necessary control of their device may find it taken away ortheir freedom limited. New MMS users need training and encouragement tosafely operate their systems.

In some embodiments, one or more cameras may be located on the S-MMS 18.These cameras may be used to provide the user with a view of the areasaround and behind them (e.g. via a display that is also mounted on theS-MMS 18). In this example, image data is communicated to the S-MMScontroller 110B, such as via a communication interface of the CNI 371,and the S-MMS controller 110B generates an image display via video orstill images to the display. Alternately, one or more displays may beconnected directly to one or more cameras. Moreover, based on ambientconditions, the image display may be filtered or otherwise modified bythe S-MMS controller 110B for purposes of enhancing user situationalawareness. Examples of such modification by the S-MMS controller 110Binclude filters that highlight contrasts, transitions in the image,and/or filters such as infrared filters to assist with night driving.

As previously disclosed (FIG. 21), the S-MMS 18 may be configured topair with virtual reality (VR) or augmented reality (AR) systems. In oneembodiment, this capability is used to provide safe, secure training.Training scenarios or programs may be either pre-programmed (e.g.included in memory 120 of the S-MMS 18) or downloaded to the device fromone or more remote servers 1510 as desired. In some embodiments, thesescenarios may utilize specific features and capabilities of the S-MMScontroller 110B, motor controller 351 and/or HMI 352 accessed via an API135. Data related to the training plan, progress, and level of the usermay be stored in a secure memory 120 on the S-MMS 18 and on a securememory on a remote server 1510 in some embodiments. Additionally, dataassociated with the user may be associated with a unique individual userprofile. The training programs may allow unique behavior of S-MMS 18controls while still maintaining and being subservient to the criticalsafety systems (e.g. collision avoidance 526 and stability management525) of the SAC 302B.

A training simulation may involve working the controls (e.g. HMI 352) ofthe S-MMS 18 to complete exercises. In an embodiment, the gameapplication may consist of game logic hosted on a complex applicationspace 130 which is set up by a parent, or otherwise configured based ona desired goal. The S-MMS 18, in some scenarios in concert with anapplication running on a smart device 1502 and/or sensory device(s,)such as a headset 2102, is guided by the user through an augmented orvirtual world to achieve a desired outcome, training, or encouragement.In some embodiments, with an AR device 2102, a user could drive a courseand collect virtual coins that they see in AR, or navigate to avoidvirtual obstacles like cones, puddles, alligators, chairs, etc. that areprojected on the real environment inside the AR device 2102. The S-MMScontroller 110 may respond to virtual objects like it would thoseobjects in the real world, allowing the user to learn safe proximity toobjects in all directions. The training could progress in difficulty,allow users to achieve different levels, or encourage repeatingexercises or training levels that are not successfully completed. Insome embodiments, the achievement of levels or tasks in an augmented orvirtual training simulation may unlock new real features, settings, orperformance levels on the user's S-MMS. As a non-limiting example, theS-MMS 18 maximum speed may increase after completing a predefined numberof levels.

In some embodiments, these or similar AR and VR programs may be used toperiodically test a user's capabilities over time to analyze for anychanges in their overall skill level, reaction time, and other metrics.This data may be very useful in analyzing changes in a user's overallhealth and ability especially, for instance, if the results of the testsdegrade regularly over time. User result data may be stored andassociated with an individual user profile and may be treated as ePHI.

AR may additionally or alternatively be used as a behavioral incentivesystem. S-MMS 18 users may be encouraged to navigate to locations byhaving a target set by another person or system. As a non-limitingexample, if a parent 1544 wanted their child to drive an S-MMS 18 towardtheir van so they could go to school, the parent could set the van asthe target either via a web interface 1542 or smart device 1532, and theuser would see (via AR) a path to the van. The user could navigate, withthe S-MMS controller 110B correcting back to the path determined by thedrive path manager 529 of the SAC 302B with a predetermined amount ofassistance, or no assistance, depending on settings and preferences. Ascore may be kept and rewards or feedback given during the trip or atthe destination for levels of achievement based on how well the usercontrolled the S-MMS 18 compared to an ideal path as determined by thedrive path manager 529.

These same or similar AR and VR capabilities may be utilized for generalgaming. Socialization may be a challenge for MMS users. Inclusion,contribution, and community all derive from, or are reliant on,socialization of some kind. Games like Pokémon Go show that gamingoutside the home is a real, powerful, communal activity. Gaming with anS-MMS 18 offers opportunities for S-MMS users and non-users to share incommon activities in the same environment. However, location-basedgaming relies on the accessibility of the location to the player. TheSAC's 302B real-time understanding of the accessibility of the S-MMSgamer's surroundings allows a game hosted on the S-MMS 18 to modifygameplay so all game elements are available to the player within theaccessible area proximate to the S-MMS. Accessibility awareness opens upthe game to all users equally, ensuring complete inclusion in the gamecommunity.

While the safety systems of the disclosed S-MMS controller 110B maintainsituational awareness of the surroundings, the user may have limitedaccess to this awareness until a S-MMS controller action is triggered.Providing enhanced user awareness of potential accessibility or safetyissues provides them with more information about their surroundingssooner.

Early awareness of potential stability, collision, and/or health issueshelps the user grow in independence by relying less on the S-MMScontroller 110B to intervene. In addition, it may allow users to createsafer plans and more efficient manual navigation routes and maneuvers.Awareness may be delivered to the user in many ways via the HMI 352,including through voice assistants like Siri™, or as visual cuesoverlaid on the real world through AR. Paired AR equipment (FIG. 21)allows the display of visual warnings overlaid on the real environment.These warnings may take many forms, including highlighting orspotlighting dangerous conditions, displaying preferred paths, ordisplaying navigation instructions to a predetermined location. In oneembodiment, paired AR glasses are used to display an overlay of thesituational awareness map, maintained by the tactical manager 527 of theSAC 302B, onto the view directly in front of the user. Additionally oralternatively, one or more routes determined by the drive path manager529 may be displayed on the AR glasses. A critical use is to provideinformation about unsafe conditions before the actual danger orcondition is imminent. In some embodiments, only data deemed critical tothe user (e.g. by threat assessor 528) is displayed. This may includethe ability to reduce (rather than add) the amount of visibility andinformation available to the user. Additionally or alternatively, byaccelerating the display of concerns, users who are driving fast S-MMSs18 or those with cognitive impairments that delay reaction time would beable to better navigate.

In an embodiment, in concert with turn and look mode, connectedwearables may be utilized to assist with navigation tasks. An example ofthis is the pairing of smart glasses such as Google Glass 2102 with anS-MMS 18M. When the user turns their head, the S-MMS controller 110B mayautomatically pivot the S-MMS 18M to match where the user is looking.While eventually items such as Google Glass may be common, manyindividuals may want a less conspicuous solution. For those users, thedisclosed system may be configured work with “Navigation Accessories”.These accessories may include things like baseball caps, hair ribbons,or other accessories with embedded IMU sensors and wireless transceiversthat allow similar navigation functionality with discrete, stylishconcealment.

To facilitate the understanding of the embodiments described herein, anumber of terms are defined below. The terms defined herein havemeanings as commonly understood by a person of ordinary skill in therelevant art. Terms such as “a,” “an,” and “the” are not intended torefer to only a singular entity, but rather include the general class ofwhich a specific example may be used for illustration. The terminologyherein is used to describe specific embodiments, but their usage doesnot delimit the disclosure, except as set forth in the claims.

The term “circuit” means at least either a single component or amultiplicity of components, either active and/or passive, that arecoupled together to provide a desired function. Terms such as “wire,”“wiring,” “line,” “signal,” “conductor,” and “bus” may be used to referto any known structure, construction, arrangement, technique, method,and/or process for physically transferring a signal from one point in acircuit to another. Also, unless indicated otherwise from the context ofits use herein, the terms “known,” “fixed,” “given,” “certain”, and“predetermined” generally refer to a value, quantity, parameter,constraint, condition, state, process, procedure, method, practice, orcombination thereof that is, in theory, variable, but is typically setin advance and not varied thereafter when in use.

Conditional language used herein, such as, among others, “can,” “might,”“may,” “e.g.,” and the like, unless specifically stated otherwise, orotherwise understood within the context as used, is generally intendedto convey that certain embodiments include, while other embodiments donot include, certain features, elements, and/or states. Thus, suchconditional language is not generally intended to imply that features,elements, and/or states are in any way required for one or moreembodiments or that one or more embodiments necessarily include logicfor deciding, with or without author input or prompting, whether thesefeatures, elements and/or states are included or are to be performed inany particular embodiment.

Communication between various systems and devices is disclosed herein.Communication may occur using wired and/or wireless communicationmethods and protocols including, but not limited to, cellular, 802.11,Wi-Fi, 802.15, Bluetooth, 802.20, and WiMAX.

Non-Transitory Computer Readable Medium

The various operations of methods described above may be performed byany suitable means capable of performing the operations, such as varioushardware and/or software component(s), circuits, and/or module(s).

The various illustrative logical blocks, modules, and circuits describedin connection with the present disclosure may be implemented orperformed with a hardware processor, a digital signal processor (DSP),an application specific integrated circuit (ASIC), a field programmablegate array signal (FPGA) or other programmable logic device (PLD),discrete gate or transistor logic, discrete hardware components, orcombinations thereof designed to perform the functions described herein.A hardware processor may be a microprocessor, commercially availableprocessor, controller, microcontroller, or state machine. A processormay also be implemented as a combination of two computing components,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration.

In one or more aspects, the functions described may be implemented insoftware, firmware, or any combination thereof executing on a hardwareprocessor. If implemented in software, the functions may be stored asone or more executable instructions or code on a non-transitorycomputer-readable storage medium. A computer-readable storage media maybe any available media that can be accessed by a processor. By way ofexample, and not limitation, such computer-readable storage media cancomprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othermedium that can be used to store executable instructions or otherprogram code or data structures and that can be accessed by a processor.Disk and disc, as used herein, includes compact disc (CD), laser disc,optical disc, digital versatile disc (DVD), floppy disk and Blu-ray discwhere disks usually reproduce data magnetically, while discs reproducedata optically with lasers. Combinations of the above should also beincluded within the scope of computer-readable media.

The methods disclosed herein comprise one or more steps or actions forachieving the described method. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isspecified, the order and/or use of specific steps and/or actions may bemodified without departing from the scope of the claims. Processes orsteps described in one implementation can be suitably combined withsteps of other described implementations.

Certain aspects of the present disclosure may comprise a computerprogram product for performing the operations presented herein. Forexample, such a computer program product may comprise a computerreadable storage medium having instructions stored (and/or encoded)thereon, the instructions being executable by one or more processors toperform the operations described herein.

Software or instructions may be transmitted over a transmission medium.For example, if the software is transmitted from a website, server, orother remote source using a coaxial cable, fiber optic cable, twistedpair, digital subscriber line (DSL), or wireless technologies such asinfrared, radio, and microwave, then the coaxial cable, fiber opticcable, twisted pair, DSL, or wireless technologies such as infrared,radio, and microwave are included in the definition of transmissionmedium.

Further, it should be appreciated that modules and/or other appropriatemeans for performing the methods and techniques described herein can bedownloaded and/or otherwise obtained by a user terminal and/or basestation as applicable. For example, such a device can be coupled to aserver to facilitate the transfer of means for performing the methodsdescribed herein. Alternatively, various methods described herein can beprovided via storage means (e.g., RAM, ROM, a physical storage mediumsuch as a compact disc (CD) or floppy disk, etc.), such that a terminaland/or base station can obtain the various methods upon coupling orproviding the storage means to the device.

For the sake of convenience, the operations are described as variousinterconnected functional blocks or distinct software modules. This isnot necessary, however, and there may be cases where these functionalblocks or modules are equivalently aggregated into a single logicdevice, program, or operation with unclear boundaries. In any event, thefunctional blocks and software modules or described features can beimplemented by themselves or in combination with other operations ineither hardware or software.

Having described and illustrated the principles of the systems, methods,processes, and/or apparatuses disclosed herein in a preferred embodimentthereof, it should be apparent that the systems, methods, processes,and/or apparatuses may be modified in arrangement and detail withoutdeparting from such principles. Claim is made to all modifications andvariation coming within the spirit and scope of the following claims.

What is claimed is:
 1. A motorized mobile system (MMS) comprising: afirst sensor to generate first sensor data about an object with respectto a first sensor reference frame, the first sensor data about theobject comprising a first range measurement to the object and a firstbearing measurement to the object, the first range measurement having anassociated first uncertainty, and the first bearing measurement havingan associated second uncertainty; a second sensor to generate secondsensor data about the object with respect to a second sensor referenceframe, the second sensor data about the object comprising a second rangemeasurement to the object and a second bearing measurement to theobject, the second range measurement having an associated thirduncertainty, and the second bearing measurement having an associatedfourth uncertainty; and a processor to: receive the first sensor dataabout the object; receive the second sensor data about the object;responsive to receiving the first and second sensor data, match thefirst and second sensor data by time; responsive to matching the firstand second sensor data by time, determine whether one or more of thematching first and second sensor data comprises polar coordinates;responsive to determining one or more of the matching first and secondsensor data comprises polar coordinates, convert the one or morematching first and second sensor data comprising polar coordinates toCartesian coordinates; translate the first sensor data from the firstsensor reference frame to a Cartesian coordinate system of a motorizedmobile system reference frame; translate the second sensor data from thesecond sensor reference frame to the Cartesian coordinate system of themotorized mobile system reference frame; select a lower rangeuncertainty between the first uncertainty and the third uncertainty;select a lower bearing uncertainty between the second uncertainty andthe fourth uncertainty; and combine the bearing measurement associatedwith the selected lower bearing uncertainty and the range measurementassociated with the selected lower range uncertainty as a location ofthe object within a reduced area of uncertainty in the Cartesiancoordinate system of the motorized mobile system reference frame.
 2. Thesystem of claim 1 wherein the processor uses the location of the objectwithin the reduced area of uncertainty for at least one navigationoperation.
 3. The system of claim 1 wherein the processor uses thelocation of the object within the reduced area of uncertainty forobstacle avoidance.
 4. The system of claim 1 wherein the processoridentifies the location of the object within the reduced area ofuncertainty on a situational awareness map maintained by the processor.5. The system of claim 1 wherein the first uncertainty is larger thanthe second uncertainty, and the fourth uncertainty is larger than thethird uncertainty.
 6. The system of claim 1 wherein the processorretrieves the first, second, third, and fourth uncertainties from amemory.
 7. The system of claim 1 wherein uncertainty comprises knownerror in a physical measurement.
 8. The system of claim 1 wherein themotorized mobile system comprises a mobile chair, a mobility scooter, anelectronic conveyance vehicle, a riding lawn mower, a grocery cart, anall-terrain vehicle, an off-road vehicle, or a golf cart.
 9. The systemof claim 1 wherein the object comprises at least one of a physicalthing, a person, an animal, a ground feature, and a surface condition.10. The system of claim 1 wherein the first sensor comprises at leastone of an optical sensor, a sound sensor, a hall effect sensor, aproximity sensor, a radar sensor, a sonar sensor, an ultrasonic sensor,a LIDAR sensor, and a camera capable of distance detection.
 11. Thesystem of claim 1 wherein the second sensor comprises at least one of anoptical sensor, a sound sensor, a hall effect sensor, a proximitysensor, a radar sensor, a sonar sensor, an ultrasonic sensor, a LIDARsensor, and a camera capable of distance detection.
 12. The system ofclaim 1 wherein the first range measurement comprises a distance to theobject from a point of operation of the first sensor.
 13. The system ofclaim 1 wherein the first bearing measurement comprises a direction tothe object from a point of operation of the first sensor.
 14. The systemof claim 13 wherein the direction to the object is expressed in degreesoffset from a true baseline direction.
 15. The system of claim 1 whereinthe second range measurement comprises a distance to the object from apoint of operation of the second sensor.
 16. The system of claim 1wherein the second bearing measurement comprises a direction to theobject from a point of operation of the second sensor.
 17. The system ofclaim 16 wherein the direction to the object is expressed in degreesoffset from a true baseline direction.
 18. The system of claim 1 whereinthe first uncertainty comprises a parameter, associated with a result ofthe first range measurement, that characterizes a dispersion of valuesthat could reasonably be attributed to the first range measurement. 19.The system of claim 1, wherein the second uncertainty comprises aparameter, associated with a result of the first bearing measurement,that characterizes a dispersion of values that could reasonably beattributed to the first bearing measurement.
 20. The system of claim 1wherein the third uncertainty comprises a parameter, associated with aresult of the second range measurement, that characterizes a dispersionof values that could reasonably be attributed to the second rangemeasurement.
 21. The system of claim 1, wherein the fourth uncertaintycomprises a parameter, associated with a result of the second bearingmeasurement, that characterizes a dispersion of values that couldreasonably be attributed to the second bearing measurement.
 22. Thesystem of claim 1 wherein at least one of the first and second sensorsgenerating data about the object is located on a vehicle other than themotorized mobile system.
 23. The system of claim 1 wherein a target isassociated with the object, and the first sensor measures a range and abearing to the target associated with the object, generates the range tothe target as the first range measurement to the object, and generatesthe bearing to the target as the first bearing measurement to theobject.
 24. The system of claim 23 wherein the target comprises at leastone of an optical target, a retro-reflective target, a graphic, asticker, and a decal.
 25. The system of claim 1 wherein a target isassociated with the object, and the second sensor measures a range and abearing to the target associated with the object, generates the range tothe target as the second range measurement to the object, and generatesthe bearing to the target as the second bearing measurement to theobject.
 26. The system of claim 25 wherein the target comprises at leastone of an optical target, a retro-reflective target, a graphic, apattern, a sticker, and a decal.
 27. A motorized mobile system (MMS)comprising: a first sensor to generate first sensor data about an objectwith respect to a first sensor reference frame, the object locatedproximate to the motorized mobile system, wherein the first sensor dataabout the object comprises a first range measurement to the object and afirst bearing measurement to the object, the first range measurement hasan associated first uncertainty, and the first bearing measurement hasan associated second uncertainty; a second sensor to generate secondsensor data about the object with respect to a second sensor referenceframe, the object located proximate to the motorized mobile system,wherein the second sensor data about the object comprises a second rangemeasurement to the object and a second bearing measurement to theobject, the second range measurement has an associated thirduncertainty, and the second bearing measurement has an associated fourthuncertainty; and a processor to: receive the first sensor data about theobject; receive the second sensor data about the object; responsive toreceiving the first and second sensor data, match the first and secondsensor data by time; responsive to matching the first and second sensordata by time, determine whether one or more of the matching first andsecond sensor data comprises polar coordinates; responsive todetermining one or more of the matching first and second sensor datacomprises polar coordinates, convert the one or more matching first andsecond sensor data comprising polar coordinates to Cartesiancoordinates; translate the first sensor data from the first sensorreference frame to a Cartesian coordinate system of a motorized mobilesystem reference frame; translate the second sensor data from the secondsensor reference frame to the Cartesian coordinate system of themotorized mobile system reference frame; select a lower rangeuncertainty between the first uncertainty and the third uncertainty;select a lower bearing uncertainty between the second uncertainty andthe fourth uncertainty; and combine the bearing measurement associatedwith the selected lower bearing uncertainty and the range measurementassociated with the selected lower range uncertainty as a location ofthe object within a reduced area of uncertainty in the Cartesiancoordinate system of the motorized mobile system reference frame. 28.The system of claim 1 wherein the processor converts the one or morematching first and second sensor data comprising polar coordinates toCartesian coordinates using a standard conversion comprising:x _(m) =r _(m) cos θ_(m) and y _(m) =r _(m) sin θ_(m), wherein r_(m) isa range to the object in a polar coordinate system, θ_(m) is a bearingof the object in the polar coordinate system, x_(m) is a rangecoordinate of the object in the Cartesian coordinate system of themotorized mobile system reference frame, and y_(m) is a bearingcoordinate of the object in the Cartesian coordinate system of themotorized mobile system reference frame.
 29. The system of claim 1further comprising an inertial measurement unit to provide one or moreof orientation, attitude, and heading to the processor, wherein theprocessor uses the one or more of orientation, attitude, and heading toconvert the one or more matching first and second sensor data comprisingpolar coordinates to the Cartesian coordinates.